How to migrate website to Amazon EC2 and RWS
We have just migrated a site from Windows Server to Amazon RDS and EC2. Basically, we needed to move to the cloud separating database from the web server. RDS now runs MySQL 5.7 and EC2 runs on Amazon Linux with Apache. Before the start, on our Windows server, we installed putty to be able to connect to the web server and WinSCP to transfer all the files.
Initial desicion was to keep MySQL, but when choosing between Linux distributions, Amazon Linux seemed like a good option. It is based on CentOS, however, it has been heavily modified to suit AWS. After reading quite a bit about it and looking at different benchmark case studies, it looked like a good choice. It is fast and responsive. It is built and maintained by Amazon for better integration with management tools pre-installed. It is getting updated on a 6-month release cycle and upgrading usually goes without any problems. However, this may be not the best option if you need special packages and community support. It took us some time to decide between AL and Ubuntu.
Step 1, Set Up and Adding Security Groups.
Reference http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SettingUp.html if needed to sign up, create user and login. Once logged in, select the region where our servers will reside in the top right corner, in our case, Oregon. We now need to create two security groups, one used for RDS database instance and one for EC2 web instance. Click home button and navigate to “VPC” under “Networking”.
Go to “Security Groups” -> “Create Security Group” to create new security group for our RDS instance. Give it a name and description, such as site-name-rds. Select VPC and click create.
Select created group and edit inbound rules. We need to add ip address from which we will be connecting, which is the server we will be transferring database from.
Your ip will be masked as XXX.XXX.XXX.XXX, replace as needed. Select MySQL/Aurora (3306), add XXX.XXX.XXX.XXX/32 ip address. It is followed by /32 for exact match ip. Use XXX.XXX.XXX.0/24 for a range of all ip addresses in the subnet.
We will later add one more ip of EC2 web instance to this group.
Once again, go to “Security Groups” -> “Create Security Group” to create new security group for our EC2 instance. Give it a name and description, such as site-name-web. Select VPC and click create.
To allow all incoming traffic over HTTP and HTTPS lets add:
HTTP(80) 0.0.0.0/0 and HTTPS(443) 0.0.0.0/0.
Next, we need to be able to SSH, so lets add:
SSH(22) XXX.XXX.XXX.XXX/32
Save.
Step 2, Set Up RDS Instance.
Reference http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.MySQL.html if needed.
Make sure that the right region where our servers will reside is selected in the top right corner, in our case, Oregon. Click home button and navigate to “RDS” under “Database”.
Click Launch DB Instance. Select MySQL and then MySQL version. We will use 5.7.11, db.t2.small that will give us 1 vCPU and 2Gib RAM. Since this is a production instance, we will use Multi-AZ Deployment for maximum availability. We will also use 12GB on General Purpose SSD for our databases.
Next, we name our insance site-name-cloud-db, chose db username and password and proceed to “Next Step”. Here we pick the right VPC, Subnet Group, leave it publicly accessible and pick our site-name-rds security group. We leave everything else as is, changing only backup window to 14 days.
Launch DB Instance. It is now being created. You can click “View Your DB Instances” and watch the status. Once changed to “available”, we can start using it.
Step 3, Set Up EC2 Instance.
Reference http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Tutorials.WebServerDB.CreateWebServer.html if needed.
Make sure that the right region where our servers will reside is selected in the top right corner, in our case, Oregon. Click home button and navigate to “EC2” under “Compute”.
Before we begin, lets get a key pair that we will need to connect to the server once it is set up. For reference, use http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html#create-a-key-pair . On the left side menu, under “network & security” click “Key Pairs”. Click create key pair, give it a name, such as my-site-key-pair, click create and it will be automatically downloaded. Save it in the safe location on your machine.
Next, click on “Instances” on the left menu and click “Launch Instance”.
Select Amazon Linux AMI. For our simple server we pick t2.micro with 1 vCPU and 1GiB of Memory.
Click “Next” to configure instance details, where we left everything default. On the next screen we changed the storage to suit our needs, which is 100GiB general purpose SSD. On the next screen we create a tag of site-name-web and move forward. Next screen gives us an option to pick existing security group site-name-web create earlier.
Click Launch, verify that information is correct, pick key pair created earlier and click “Launch Instances”. It is now being created. Click “View Instances” and watch the status. Once changed to “running”, we can start using it.
Step 4, Add ip of new EC2 Instance to RDS Security Group.
Run this command on EC2 server to get the ip address and get ip in inet addr under eth0
ifconfig
Edit security group for your site-name-rds and add ip or subnet as:
MySQL/Aurora (3306) 172.XX.X.0/24
Save.
Step 5, Set Up putty and Connect.
Launch PuTTYgen. Make sure SSH-2-RSA option is checked. Click “Load”, in the drop down option select “All Files”, locate the .pem file downloaded earlier after creating Key Pairs, “save private key” with the same name and .ppk extension.
Now you need to get your public dns name. Once EC2 instance is loaded, click on it and in the “Description” below copy Public DNS.
Start putty. In the Host Name field enter user@public-dns, such as ec2-user@ec2-XX-XX-XXX-XX.us-west-2.compute.amazonaws.com. Amazon Linux instances get created with ec2-user by default. Confirm that selected port is 22 and connection type is SSH.
Enter the name in “saved sessions”, such as site-name-aws and click save.
On the left hand side menu in putty navigate to Connection – SSH – Auth. Click Browse and locate .ppk key created using PuTTYgen earlier. Go back to “Session” menu, click save and then open. Click “Yes” if it displays a message and you should be on the server and ready to configure.
Step 6, Set Up Web Server.
First, lets update the packages
sudo yum update -y
Next, install apache, php 5.6 and mysql driver
sudo yum install -y httpd24 php56 php56-mysqlnd
Next, additional packages, if needed. For instance, we use gd image library and need to add it
sudo yum install php56-gd
Soap package:
sudo yum install php56-soap
Imagick, which requires php devel, php pear and a few others
sudo yum install kernel-devel gcc gcc-c++
sudo yum install php56-devel
sudo yum install php-pear # This line installs pecl as well as pear
sudo yum install ImageMagick-devel
sudo pecl install imagick
which requires us to open php.ini and append a line at the end of the file to load the library
sudo vi /etc/php.ini
press “i” to edit the file, add this line at the end
extension=imagick.so
click “esc” and “zz” while holding “shift” to save.
Lets start apache
sudo service httpd start
Change config to autostart on boot
sudo chkconfig httpd on
Add www group and current user to that group along with apache group
sudo groupadd www
sudo usermod -a -G www ec2-user
Lets change permissions on www folder, exit the server and re-login
sudo chown -R root:www /var/www
exit
This is it for the web server, however, we host a couple websites on it. We will create virtual hosts next
cd /var/www/
sudo mkdir vhosts
cd vhosts
sudo mkdir site-1(actual name of the site)
sudo mkdir site-2(actual name of the site)
sudo chown -R root:www /var/www
sudo chmod 2775 /var/www
find /var/www -type d -exec sudo chmod 2775 {} \;
find /var/www -type f -exec sudo chmod 0664 {} \;
cd /etc/httpd/conf.d
sudo touch vhost.conf
sudo vi /etc/httpd/conf.d/vhost.conf
paste and replace site-1.com and site-2.com with actual sites:
<VirtualHost *:80>
# Leave this alone. This setting tells Apache that
# this vhost should be used as the default if nothing
# more appropriate is available.
ServerName default:80
# REQUIRED. Set this to the directory you want to use for
# your “default” site files.
DocumentRoot /var/www/html
# Optional. Uncomment this and set it to your admin email
# address, if you have one. If there is a server error,
# this is the address that Apache will show to users.
#ServerAdmin you@example.com
# Optional. Uncomment this if you want to specify
# a different error log file than the default. You will
# need to create the error file first.
#ErrorLog /var/www/vhosts/logs/error_log
</VirtualHost>
<VirtualHost *:80>
# REQUIRED. Set this to the host/domain/subdomain that
# you want this VirtualHost record to handle.
ServerName site-1.com
# Optional. You can specify additional host names that
# serve up the same site. This can be top-level, domains,
# sub-domains, and can even use wildcard subdomains such
# as *.yourdomain.com – just separate each host name
# with a single space.
ServerAlias www.site-1.com
# REQUIRED. Set this to the directory you want to use for
# this vhost site’s files.
DocumentRoot /var/www/vhosts/site-1
# Optional. Uncomment this and set it to your admin email
# address, if you have one. If there is a server error,
# this is the address that Apache will show to users.
#ServerAdmin you@example.com
# Optional. Uncomment this if you want to specify
# a different error log file than the default. You will
# need to create the error file first.
ErrorLog /var/www/vhosts/logs/site-1_error_log
# REQUIRED. Let’s make sure that .htaccess files work on
# this site. Don’t forget to change the file path to
# match your DocumentRoot setting above.
<Directory /var/www/vhosts/site-1>
AllowOverride All
</Directory>
</VirtualHost>
<VirtualHost *:80>
# REQUIRED. Set this to the host/domain/subdomain that
# you want this VirtualHost record to handle.
ServerName site-2.com
# Optional. You can specify additional host names that
# serve up the same site. This can be top-level, domains,
# sub-domains, and can even use wildcard subdomains such
# as *.yourdomain.com – just separate each host name
# with a single space.
ServerAlias www.site-2.com
# REQUIRED. Set this to the directory you want to use for
# this vhost site’s files.
DocumentRoot /var/www/vhosts/site-2
# Optional. Uncomment this and set it to your admin email
# address, if you have one. If there is a server error,
# this is the address that Apache will show to users.
#ServerAdmin you@example.com
# Optional. Uncomment this if you want to specify
# a different error log file than the default. You will
# need to create the error file first.
ErrorLog /var/www/vhosts/logs/site-2_error_log
# REQUIRED. Let’s make sure that .htaccess files work on
# this site. Don’t forget to change the file path to
# match your DocumentRoot setting above.
<Directory /var/www/vhosts/site-2>
AllowOverride All
</Directory>
</VirtualHost>
Next, lets create log directory
cd /var/www/vhosts/
sudo mkdir logs
Restart apache
sudo service httpd restart
Step 7, Set Up SSL Keys and SSL Virtual Directories.
Download and install WinSCP and import connection configuration from Putty.
Export current certificates from existing site. In our case it was windows server and we used this resource for reference – https://www.sslshopper.com/move-or-copy-an-ssl-certificate-from-a-windows-server-to-an-apache-server.html that gave us site.pfx file. Upload the file to /home/ec2-user via WinSCP.
Now, in our ssh session (putty) we need to convert and store the keys:
cd /home/ec2-user/
openssl pkcs12 -in estoresbyzome.pfx -out estoresbyzome.txt -nodes
copy each key and certificate from and including opening —- , to and including closing of each section while pasting it into site.key, site.crt, and if there is more than 2 – intermidiateCA.crt. Then move them to private folder and change permissions.
sudo cp site.key /etc/pki/tls/private/site.key
sudo cp site.crt /etc/pki/tls/private/site.crt
sudo cp intermediateCA.crt /etc/pki/tls/private/intermediateCA.crt
cd /etc/pki/tls/private/
sudo chmod 600 site.key
sudo chmod 600 site.crt
sudo chmod 644 intermediateCA.crt
Install mod ssl for apache and restart
sudo yum install -y mod24_ssl
sudo service httpd restart
Next, we need to change ssl certificate and key in ssl.conf
sudo vi /etc/httpd/conf.d/ssl.conf
to
SSLCertificateFile /etc/pki/tls/private/site.crt
SSLCertificateKeyFile /etc/pki/tls/private/site.key
sudo service httpd restart
Add virtual host entries:
cd /etc/httpd/conf.d/
sudo vi vhost.conf
At the end of the file add:
<VirtualHost *:443>
ServerName site-1.com
ServerAlias www.site-1.com
DocumentRoot /var/www/vhosts/site-1
SSLEngine on
SSLCertificateFile /etc/pki/tls/private/site.crt
SSLCertificateKeyFile /etc/pki/tls/private/site.key
ErrorLog /var/www/vhosts/logs/site-1_error_log
</VirtualHost>
<VirtualHost *:443>
ServerName site-2.com
ServerAlias www.site-2.com
DocumentRoot /var/www/vhosts/site-2
SSLEngine on
SSLCertificateFile /etc/pki/tls/private/site.crt
SSLCertificateKeyFile /etc/pki/tls/private/site.key
ErrorLog /var/www/vhosts/logs/site-2_error_log
</VirtualHost>
Save and restart apache, this should be it
sudo service httpd restart
Step 8, Display Maintenance Message on Live Site.
Since our site uses the same header, displaying maintenance message is as easy as pasting html and php exit at the top of it:
<!doctype html>
<title>Site Maintenance</title>
<style>
body { text-align: center; padding: 150px; }
h1 { font-size: 50px; }
body { font: 20px Helvetica, sans-serif; color: #333; }
article { display: block; text-align: left; width: 650px; margin: 0 auto; }
a { color: #dc8100; text-decoration: none; }
a:hover { color: #333; text-decoration: none; }
</style>
<p><img src="img/logo.gif" alt="Site" title="Site" width="150" height="100" align="left"></p>
<article>
<h1>We&rsquo;ll be back soon!</h1>
<div>
<p>Sorry for the inconvenience but we&rsquo;re performing some maintenance at the moment. We&rsquo;ll be back online shortly!</p>
<p>&mdash; Site-1 Team</p>
</div>
</article>
<?php
exit;
?>
Step 9, Backup Database.
Stop the production db and start with locked tables if exit in the header doesn’t stop complete access before the backup.
mysqldump -u root -p -h 127.0.0.1 site-1-db > c:\Backups\site-1-db(build_date).sql
mysqldump -u root -p -h 127.0.0.1 site-2-db > c:\Backups\site-2-db(build_date).sql
Step 10, Stop Production Database.
Since we are running a Windows server, run this command in command line
net stop mysql
Step 11, Create, Restore the Database and Update Schema.
mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com
create database site-2;
exit;
mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com site-1-db < c:\Backups\right_backup.sql
mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com site-2-db < c:\Backups\right_backup_2.sql
We also had a couple of MyISAM tables and converted them to InnoDB, connect to instance
mysql -u zome -p -h site-cloud-db.your-database-endpoint.us-west-2.rds.amazonaws.com
login and check if there are any MyISAM
use INFORMATION_SCHEMA;
SELECT * FROM `TABLES` where engine = 'MyISAM' and (TABLE_SCHEMA = 'site-1-db' OR TABLE_SCHEMA = 'site-2-db’ ) ORDER BY `ENGINE` DESC
use site-1-db;
ALTER TABLE `myisam-table1` ENGINE=INNODB;
ALTER TABLE `myisam-table2` ENGINE=INNODB;
etc.
Step 12, Transfer Files and change db connections.
Login via WinSCP and transfer all the files to the right vhost directories. Change site configurations to point to the new database.
Step 13, change permissions to write directories for apache.
To make directories writable to apache execute the following command on the directories that need write access:
sudo chown -R root:apache directory-name
Step 14, Test.
To test before going live edit the c:\windows\system32\drivers\etc\hosts file on local machine to point to the right server by specifying ip address:
XXX.XXX.XXX.XXX site-1.com www.site-1.com
XXX.XXX.XXX.XXX site-2.com www.site-2.com
Step 15, Change DNS to Point to the New Site.
If you are using AWS for DNS, navigate to Route 53, if not, point your domain A record to the new ip address. Perform the last set of testing and the site in now on AWS.
