Ubuntu How To

We have just migrated a site from Windows Server to Amazon RDS and EC2. Basically, we needed to move to the cloud separating database from the web server. RDS now runs MySQL 5.7 and EC2 runs on Amazon Linux with Apache. Before the start, on our Windows server, we installed putty to be able to connect to the web server and WinSCP to transfer all the files.

Initial desicion was to keep MySQL, but when choosing between Linux distributions, Amazon Linux seemed like a good option. It is based on CentOS, however, it has been heavily modified to suit AWS. After reading quite a bit about it and looking at different benchmark case studies, it looked like a good choice. It is fast and responsive. It is built and maintained by Amazon for better integration with management tools pre-installed. It is getting updated on a 6-month release cycle and upgrading usually goes without any problems. However, this may be not the best option if you need special packages and community support. It took us some time to decide between AL and Ubuntu.

Step 1, Set Up and Adding Security Groups.

Reference http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SettingUp.html if needed to sign up, create user and login. Once logged in, select the region where our servers will reside in the top right corner, in our case, Oregon.  We now need to create two security groups, one used for RDS database instance and one for EC2 web instance. Click home button and navigate to “VPC” under “Networking”.

Go to “Security Groups” -> “Create Security Group” to create new security group for our RDS instance. Give it a name and description, such as site-name-rds. Select VPC and click create.

Select created group and edit inbound rules. We need to add ip address from which we will be connecting, which is the server we will be transferring database from.

Your ip will be masked as XXX.XXX.XXX.XXX, replace as needed.  Select MySQL/Aurora (3306), add XXX.XXX.XXX.XXX/32 ip address. It is followed by /32 for exact match ip. Use XXX.XXX.XXX.0/24 for a range of all ip addresses in the subnet.

We will later add one more ip of EC2 web instance to this group.

Once again, go to “Security Groups” -> “Create Security Group” to create new security group for our EC2 instance. Give it a name and description, such as site-name-web. Select VPC and click create.

To allow all incoming traffic over HTTP and HTTPS lets add:

HTTP(80) 0.0.0.0/0 and HTTPS(443) 0.0.0.0/0.

Next, we need to be able to SSH, so lets add:

SSH(22) XXX.XXX.XXX.XXX/32

Save.

Step 2, Set Up RDS Instance.

Reference http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.MySQL.html if needed.

Make sure that the right region where our servers will reside is selected in the top right corner, in our case, Oregon. Click home button and navigate to “RDS” under “Database”.

Click Launch DB Instance. Select MySQL and then MySQL version. We will use 5.7.11, db.t2.small that will give us 1 vCPU and 2Gib RAM. Since this is a production instance, we will use Multi-AZ Deployment for maximum availability. We will also use 12GB on General Purpose SSD for our databases.

Next, we name our insance site-name-cloud-db, chose db username and password and proceed to “Next Step”. Here we pick the right VPC, Subnet Group, leave it publicly accessible and pick our site-name-rds security group. We leave everything else as is, changing only backup window to 14 days.

Launch DB Instance. It is now being created. You can click “View Your DB Instances” and watch the status. Once changed to “available”, we can start using it.

Step 3, Set Up EC2 Instance.

Reference http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Tutorials.WebServerDB.CreateWebServer.html if needed.

Make sure that the right region where our servers will reside is selected in the top right corner, in our case, Oregon. Click home button and navigate to “EC2” under “Compute”.

Before we begin, lets get a key pair that we will need to connect to the server once it is set up. For reference, use http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html#create-a-key-pair . On the left side menu, under “network & security” click “Key Pairs”. Click create key pair, give it a name, such as my-site-key-pair, click create and it will be automatically downloaded. Save it in the safe location on your machine.

Next, click on “Instances” on the left menu and click “Launch Instance”.

Select Amazon Linux AMI. For our simple server we pick t2.micro with 1 vCPU and 1GiB of Memory.

Click “Next” to configure instance details, where we left everything default. On the next screen we changed the storage to suit our needs, which is 100GiB general purpose SSD. On the next screen we create a tag of site-name-web and move forward. Next screen gives us an option to pick existing security group site-name-web create earlier.

Click Launch, verify that information is correct, pick key pair created earlier and click “Launch Instances”. It is now being created. Click “View Instances” and watch the status. Once changed to “running”, we can start using it.

Step 4, Add ip of new EC2 Instance to RDS Security Group.

Run this command on EC2 server to get the ip address and get ip in inet addr under eth0

ifconfig

Edit security group for your site-name-rds and add ip or subnet as:

MySQL/Aurora (3306) 172.XX.X.0/24

Save.

Step 5, Set Up putty and Connect.

Launch PuTTYgen. Make sure SSH-2-RSA option is checked. Click “Load”, in the drop down option select “All Files”, locate the .pem file downloaded earlier after creating Key Pairs, “save private key” with the same name and .ppk extension.

Now you need to get your public dns name. Once EC2 instance is loaded, click on it and in the “Description” below copy Public DNS.

Start putty. In the Host Name field enter user@public-dns, such as ec2-user@ec2-XX-XX-XXX-XX.us-west-2.compute.amazonaws.com. Amazon Linux instances get created with ec2-user by default. Confirm that selected port is 22 and connection type is SSH.

Enter the name in “saved sessions”, such as site-name-aws and click save.

On the left hand side menu in putty navigate to Connection – SSH – Auth. Click Browse and locate .ppk key created using PuTTYgen earlier. Go back to “Session” menu, click save and then open. Click “Yes” if it displays a message and you should be on the server and ready to configure.

Step 6, Set Up Web Server.

First, lets update the packages

sudo yum update -y

Next, install apache, php 5.6 and mysql driver

sudo yum install -y httpd24 php56 php56-mysqlnd

Next, additional packages, if needed. For instance, we use gd image library and need to add it

sudo yum install php56-gd

Soap package:

sudo yum install php56-soap

Imagick, which requires php devel, php pear and a few others

sudo yum install kernel-devel gcc gcc-c++

sudo yum install php56-devel

sudo yum install php-pear # This line installs pecl as well as pear

sudo yum install ImageMagick-devel

sudo pecl install imagick

which requires us to open php.ini and append a line at the end of the file to load the library

sudo vi /etc/php.ini

press “i” to edit the file, add this line at the end

extension=imagick.so

click “esc” and “zz” while holding “shift” to save.

Lets start apache

sudo service httpd start

Change config to autostart on boot

sudo chkconfig httpd on

Add www group and current user to that group along with apache group

sudo groupadd www

sudo usermod -a -G www ec2-user

Lets change permissions on www folder, exit the server and re-login

sudo chown -R root:www /var/www

exit

This is it for the web server, however, we host a couple websites on it. We will create virtual hosts next

cd /var/www/

sudo mkdir vhosts

cd vhosts

sudo mkdir site-1(actual name of the site)

sudo mkdir site-2(actual name of the site)

sudo chown -R root:www /var/www

sudo chmod 2775 /var/www

find /var/www -type d -exec sudo chmod 2775 {} \;

find /var/www -type f -exec sudo chmod 0664 {} \;

cd /etc/httpd/conf.d

sudo touch vhost.conf

sudo vi /etc/httpd/conf.d/vhost.conf

paste and replace site-1.com and site-2.com with actual sites:


<VirtualHost *:80>

# Leave this alone. This setting tells Apache that

# this vhost should be used as the default if nothing

# more appropriate is available.

ServerName default:80

# REQUIRED. Set this to the directory you want to use for

# your “default” site files.

DocumentRoot /var/www/html

# Optional. Uncomment this and set it to your admin email

# address, if you have one. If there is a server error,

# this is the address that Apache will show to users.

#ServerAdmin you@example.com

# Optional. Uncomment this if you want to specify

# a different error log file than the default. You will

# need to create the error file first.

#ErrorLog /var/www/vhosts/logs/error_log

</VirtualHost>

 

<VirtualHost *:80>

# REQUIRED. Set this to the host/domain/subdomain that

# you want this VirtualHost record to handle.

ServerName site-1.com

# Optional. You can specify additional host names that

# serve up the same site. This can be top-level, domains,

# sub-domains, and can even use wildcard subdomains such

# as *.yourdomain.com – just separate each host name

# with a single space.

ServerAlias www.site-1.com

# REQUIRED. Set this to the directory you want to use for

# this vhost site’s files.

DocumentRoot /var/www/vhosts/site-1

# Optional. Uncomment this and set it to your admin email

# address, if you have one. If there is a server error,

# this is the address that Apache will show to users.

#ServerAdmin you@example.com

# Optional. Uncomment this if you want to specify

# a different error log file than the default. You will

# need to create the error file first.

ErrorLog /var/www/vhosts/logs/site-1_error_log

# REQUIRED. Let’s make sure that .htaccess files work on

# this site. Don’t forget to change the file path to

# match your DocumentRoot setting above.

<Directory /var/www/vhosts/site-1>

AllowOverride All

</Directory>

</VirtualHost>

 

<VirtualHost *:80>

# REQUIRED. Set this to the host/domain/subdomain that

# you want this VirtualHost record to handle.

ServerName site-2.com

# Optional. You can specify additional host names that

# serve up the same site. This can be top-level, domains,

# sub-domains, and can even use wildcard subdomains such

# as *.yourdomain.com – just separate each host name

# with a single space.

ServerAlias www.site-2.com

# REQUIRED. Set this to the directory you want to use for

# this vhost site’s files.

DocumentRoot /var/www/vhosts/site-2

# Optional. Uncomment this and set it to your admin email

# address, if you have one. If there is a server error,

# this is the address that Apache will show to users.

#ServerAdmin you@example.com

# Optional. Uncomment this if you want to specify

# a different error log file than the default. You will

# need to create the error file first.

ErrorLog /var/www/vhosts/logs/site-2_error_log

# REQUIRED. Let’s make sure that .htaccess files work on

# this site. Don’t forget to change the file path to

# match your DocumentRoot setting above.

<Directory /var/www/vhosts/site-2>

AllowOverride All

</Directory>

</VirtualHost>

Next, lets create log directory

cd /var/www/vhosts/

sudo mkdir logs

Restart apache

sudo service httpd restart

Step 7, Set Up SSL Keys and SSL Virtual Directories.

Download and install WinSCP and import connection configuration from Putty.

Export current certificates from existing site. In our case it was windows server and we used this resource for reference – https://www.sslshopper.com/move-or-copy-an-ssl-certificate-from-a-windows-server-to-an-apache-server.html that gave us site.pfx file. Upload the file to /home/ec2-user via WinSCP.

Now, in our ssh session (putty) we need to convert and store the keys:

cd /home/ec2-user/

openssl pkcs12 -in estoresbyzome.pfx -out estoresbyzome.txt -nodes

copy each key and certificate from and including opening —- , to and including closing of each section while pasting it into site.key, site.crt, and if there is more than 2 – intermidiateCA.crt. Then move them to private folder and change permissions.

sudo cp site.key /etc/pki/tls/private/site.key

sudo cp site.crt /etc/pki/tls/private/site.crt

sudo cp intermediateCA.crt /etc/pki/tls/private/intermediateCA.crt

cd /etc/pki/tls/private/

sudo chmod 600 site.key

sudo chmod 600 site.crt

sudo chmod 644 intermediateCA.crt

Install mod ssl for apache and restart

sudo yum install -y mod24_ssl

sudo service httpd restart

Next, we need to change ssl certificate and key in ssl.conf

sudo vi /etc/httpd/conf.d/ssl.conf

to


SSLCertificateFile /etc/pki/tls/private/site.crt

SSLCertificateKeyFile /etc/pki/tls/private/site.key

sudo service httpd restart

Add virtual host entries:

cd /etc/httpd/conf.d/

sudo vi vhost.conf

At the end of the file add:


<VirtualHost *:443>

ServerName site-1.com

ServerAlias www.site-1.com

DocumentRoot /var/www/vhosts/site-1

SSLEngine on

SSLCertificateFile /etc/pki/tls/private/site.crt

SSLCertificateKeyFile /etc/pki/tls/private/site.key

ErrorLog /var/www/vhosts/logs/site-1_error_log

</VirtualHost>

 

<VirtualHost *:443>

ServerName site-2.com

ServerAlias www.site-2.com

DocumentRoot /var/www/vhosts/site-2

SSLEngine on

SSLCertificateFile /etc/pki/tls/private/site.crt

SSLCertificateKeyFile /etc/pki/tls/private/site.key

ErrorLog /var/www/vhosts/logs/site-2_error_log

</VirtualHost>

Save and restart apache, this should be it

sudo service httpd restart

Step 8, Display Maintenance Message on Live Site.

Since our site uses the same header, displaying maintenance message is as easy as pasting html and php exit at the top of it:

<!doctype html>
<title>Site Maintenance</title>
<style>
body { text-align: center; padding: 150px; }
h1 { font-size: 50px; }
body { font: 20px Helvetica, sans-serif; color: #333; }
article { display: block; text-align: left; width: 650px; margin: 0 auto; }
a { color: #dc8100; text-decoration: none; }
a:hover { color: #333; text-decoration: none; }
</style>

<p><img src="img/logo.gif" alt="Site" title="Site" width="150" height="100" align="left"></p>
<article>
<h1>We’ll be back soon!</h1>
<div>
<p>Sorry for the inconvenience but we’re performing some maintenance at the moment. We’ll be back online shortly!</p>
<p>— Site-1 Team</p>
</div>
</article>
<?php
exit;
?>

Step 9, Backup Database.

Stop the production db and start with locked tables if exit in the header doesn’t stop complete access before the backup.

mysqldump -u root -p -h 127.0.0.1 site-1-db > c:\Backups\site-1-db(build_date).sql

mysqldump -u root -p -h 127.0.0.1 site-2-db > c:\Backups\site-2-db(build_date).sql

Step 10, Stop Production Database.

Since we are running a Windows server, run this command in command line

net stop mysql

Step 11, Create, Restore the Database and Update Schema.

mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com

create database site-2;

exit;

mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com site-1-db < c:\Backups\right_backup.sql

mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com site-2-db < c:\Backups\right_backup_2.sql

We also had a couple of MyISAM tables and converted them to InnoDB, connect to instance

mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com

login and check if there are any MyISAM

use INFORMATION_SCHEMA;

SELECT * FROM `TABLES` where engine = 'MyISAM' and (TABLE_SCHEMA = 'site-1-db' OR TABLE_SCHEMA = 'site-2-db’ ) ORDER BY `ENGINE` DESC

use site-1-db;

ALTER TABLE `myisam-table1` ENGINE=INNODB;

ALTER TABLE `myisam-table2` ENGINE=INNODB;

etc.

Step 12, Transfer Files and change db connections.

Login via WinSCP and transfer all the files to the right vhost directories. Change site configurations to point to the new database.

Step 13, change permissions to write directories for apache.

To make directories writable to apache execute the following command on the directories that need write access:
sudo chown -R root:apache directory-name

Step 14, Test.

To test before going live edit the c:\windows\system32\drivers\etc\hosts file on local machine to point to the right server by specifying ip address:
XXX.XXX.XXX.XXX site-1.com www.site-1.com
XXX.XXX.XXX.XXX site-2.com www.site-2.com

Step 15, Change DNS to Point to the New Site.

If you are using AWS for DNS, navigate to Route 53, if not, point your domain A record to the new ip address. Perform the last set of testing and the site in now on AWS.

This is it

 

<edit> I have purchased Web Hosting Deals site several years ago, just couldn’t get to it yet. Still in it’s original form. I am planning to redo the whole site at some point, sign up with more hosting companies and do complete feature reviews of linux hosting companies.

Meanwhile, please feel free to comment and let us know which Linux hosts you use and why.

As you all may know, the new version of Ubuntu was released on October 13, 2011, which is just over a week ago as of the time of writing this post. I wanted to upgrade right away, but decided to wait a bit for the user response as upgrades don’t always go smooth. Well, this one worked like a charm for me.

It took a little over an hour to perform the full upgrade. I made a backup just in case, which I also recommend doing. So far just a few programs need to be reinstalled. A ton of packages got deleted, but it’s mostly because they have a newer and improved versions.

As for the changes, the first thing I noticed was the desktop. It does look a lot better due to improved unity shell. I remember how much I hated it with the last upgrade when it just got added to Ubuntu and I am still not a big fan of it. It is, however, a lot more convenient and clean. I went from “definitely removing” to “may be I’ll poke around and keep it for a while”. I also don’t like the dock sitting on the right side, but I still have to look if I can relocate and change the size of it. The dash interface became a lot more usable too.

There are a few other small, but noticeable improvements, such as a new gnome login screen, login manager now uses LightDM.  Overall performance got slightly improved. Default mail client is now mozilla Thunderbird. Ubuntu now has a better support of 32 bit apps on 64 bit systems. It has a new backup client, which I still have to look into as it looks like they give you 5gb of online storage for free. Improved software center looks a lot better. These are just a few changes that I was pleased with.

There are other changes, but I think these are the biggest ones, at least for me. I like where it is going and will be waiting patiently for the next release. I want to see more improvements to unity as it looks like they put more focus on it now and it is here to stick.

So, just an update to previous post regarding ubuntu screen flickering, I was not able to completely get rid of flickering after 5 hours of trying. I tried a dozen of solutions after searching all over Ubuntu forums and none of them resolved it for me. I gave up.

In the end I just plugged in my ati radeon hd 4870 and it worked like a charm from start. I ended up updating the drivers manually from ATI and it works even better now.

If you are a hardcore Ubuntu user, you are probably very proud of it. Most of the people I met who use Linux love to show off their systems. Their desktops scream “Check how hot I am” with sexy and geeky wallpapers, useful plug-ins, nicely organized icons and custom color schemes. These people talk about Linux non-stop, always eager to learn and share something new and useful.

Everybody have nice wallpapers, however, not everybody use avatars that let other people know what they are into without asking. Avatars are generally used on messengers, forums, social networks and blogs. Perhaps it is time to fire your pidgin and change the image you have been using for years to something newer and more relevant.

Check out these cool Ubuntu avatars, I am pretty sure you would find something that will fit your taste. If not, you can always create an avatar of your own as they have other options, such as creating one from your own image.

Just thought I would share just a few great but not the best ones I found. There are over 100 of them, these are just 6 of my quick picks 🙂

As you may have heard, the new version of Ubuntu – 10.10, codenamed Maverick Meerkat, is officially out. As a matter of fact this 10.10 version was out on 10.10.10 at 10:10:10. 🙂

First, what does it mean for this blog since it is named after an “old” version? Not much. This blog is now a general Ubuntu blog, and I will be still posting things that I find interesting about Ubuntu. Lucid Lynx was sort of an entry point for me and I got a little carried away with some of the projects that I have been working on. Initially, I was planning to post all the issues with Lucid, such as helping resolve various driver problems, etc. but I did not get too much to complain since everything worked great out of the box for me.

Second, lets look at some changes at Maverick Meerkat. 🙂

Ubuntu 10.10 has been optimized a lot for netbooks, I would like to see more Ubuntu powered netbooks hit the market. As an addition it started to support multi-touch screens.

There also have been a lot of changes to the installer, they just can’t leave it along I guess, making it better with every release. You can now download additional packages that you may need during the initial install phase. It also installs quite a bit faster now.

The desktop has been updated to the Gnome 2.32 and by default Ubuntu comes with a new wallpaper. The overall look has been noticeably changed, even the font is new.

The last change that I want to mention is paid apps platform. A lot of users seem to be pretty excited about it as it may open the doors for the development in monetary interest in Ubuntu, but I am not too excited about it since this was the primary reason I switched to Ubuntu, but I don’t complain. As a matter of fact, the whole Software Center got revamped. It has a few additional features.

Here is a new desktop for you:

Just came across this video and though it was pretty funny. Oh, things that people can do with Ubuntu sometimes are just unimaginable. 🙂

I was going to record my own video, but there are already so many out there that I decided to save some time and include the one of many that I found on youtube. Compiz consists of a few packages that you can install, this video shows examples from the standard compizconfig-settings-manager and compiz-fusion-extra-plugins. While compiz is not really new, it is still a lot of fun to play around with. BTW, this was THE reason I installed Ubuntu a few years ago.

Here is a video:

It just looked so cool, but gets old once you play around for a bit. It is still fun to show to the friends when they come over. Especially to Mac-heads since they are always so proud of their apples, which by the way is built on Linux anyway. We would constantly have Windows-Mac arguments with no outcome, however, once they see Ubuntu, they have nothing to say. Viruses – same results, the look and effects – Ubuntu wins, load time – Lucid Lynx is fast here as well, applications – Ubuntu with Wine wins. 🙂 I love the way Linux uses the consoles, allowing to have a few desktops at the same time. It is so convenient, especially when you code and have one desktop for one project. Makes multitasking so much easier. Ubuntu seems a lot faster than anything I have tried so far. Seems like I can get done a lot more, a few seconds faster here, a few there adds up a lot of extra time to put into something useful instead of just waiting for some app or load/reboot. Here is another video I just found on the load time comparison between Lucid Lynx and Windows 7. Even though it is not precise, I still found it pretty useful.

Anyways, there are too many videos on Ubuntu out. Just want to say that Lucid Lynx is perfect for my second pc.

P.S. I found out about compiz after seeing my friend’s pc. He is a hardcore FreeBSD guy and showed me Beryl over 3 years ago. I was really impressed and a few month later I found Ubuntu and Compiz. I am not sure which one came out first, Beryl or Compiz, but they seem to be very close.

I decided to start this blog about Ubuntu Lucid Lynx quite a bit ago, however, got really busy and kept on delaying the project. Well, today I finally got around to installing WordPress, configuring it and getting appropriate theme.  Looking for a theme took quite a bit of time and I even started to think about making my own when I accidentally stumbled across the theme you see right now on Ubuntu wiki page. There are quite a few other Linux themes, however, I believe this looks the closest and makes you feel right at home. 🙂 Besides, all the other Linux themes had funky looking penguins all over, why can’t they just use a regular one. You know, the one that doesn’t drink from a Windows juice box? 🙂

Anyway, I got a start! As for now, I still have a lot of work to do. There are still a lot of WordPress plug-ins to install and articles to write. I hope I can make this blog interesting and be able to teach some of you as I learn Lucid Lynx myself. I will be installing it and writing as I go. I used Ubuntu a year ago or so (which was the coolest thing since sliced bread, but didn’t have decent drivers for my new hardware), but I heard some things may have changed and I hope to find a lot of changes in Lucid Lynx.

I hope you like it, stay tuned! 🙂