The Mobile Cloud Project, Part 1.1: Basic AWS

I. Overview

The Mobile Cloud Project is my proof-of-concept for developing and hosting a scalable mobile application platform on the Amazon Web Services cloud..

Part 1.1 of this project provides a basic overview of Amazon Web Services and describes my procedure for building Ubuntu Linux virtual servers (Reserved and On-Demand instances) on Amazon EC2.

Please see the main project page for more information on the other parts of this project.

II. Consulting

I do my best to explain the concepts and techniques behind my projects. If you like my work and can use my expertise in your projects, I am available for consulting at a competitive rate.

III. Overview of Amazon Web Services

Amazon Web Services (AWS) is a collection of technologies that enable us to host our applications on Amazon’s server and network infrastructure (the "cloud"). AWS can provide an individual or startup with the potential computing power of thousands of networked servers, petabytes of storage space, and fault-tolerant capabilities, which rival the infrastructure of large corporations that possess multiple data centres and IT departments. You require no physical hardware, network infrastructure, electricity, nor legions of IT staff, in order to run your powerful virtual enterprise in the cloud. The scalability of AWS makes it an ideal solution for hosting the back-end platform for popular mobile apps used by millions of users.

1. Amazon EC2

An "instance", or virtual server, is the basic unit of computing resource in the Amazon Elastic Compute Cloud (EC2). You can create as many instances as you need, and pricing is based on instance type, "size", and hourly usage. A Reserved Instance is a dedicated virtual server which is often kept running all the time, while an On-Demand instance is usually turned on only as required.

2. Amazon S3

Amazon Simple Storage Service (S3) is a virtual data storage area. A "bucket" is analogous to a filesystem that can store objects or files. You can create as much data storage as you need in S3, and pricing is based on storage size and amount of data transferred.

3. Elastic IP Addresses

An Elastic IP address is an Internet-routable static IP address that is allocated to your AWS EC2 account. You can map an Elastic IP address to any running instance (which has a private IP address), and quickly remap to a different instance if the other instance fails. Network address translation is enabled automatically via 1:1 NAT. There is no charge for elastic IP addresses while in use.

IV. Procedure

In this procedure, I will demonstrate how to create two EC2 instances: a Reserved Instance called “Galactica” (which will eventually host this blog and related web apps) and an On-Demand instance called “Pegasus” for future use.

1. Sign up for a free AWS account

1. Go to http://aws.amazon.com and click "Sign Up Now".
2. Fill out the requested information.

2. Sign up for Amazon EC2 and S3

1. Go to http://aws.amazon.com/ec2 and click "Sign Up for Amazon EC2".
2. Review the pricing details and enter your credit card and billing information.
3. Complete the identity verification by telephone.
4. You are now signed up for both EC2 and S3.

3. Sign in to the AWS Management Console

1. Go to http://aws.amazon.com/console and click "Sign in to the AWS Console" (EC2).

4. Create Key Pairs

You will need to create a key pair for each instance you plan to launch. This allows you to access your instance securely via SSH (Linux) or RDP (Windows).

1. In Windows Explorer, create the folders “C:\aws” and “C:\aws\ec2″
2. Click "Key Pairs" in the left menu, click "Create Key Pair", and enter a name for this key pair (e.g. "Galactica").
3. Save the “Galactica.pem” private key file in the C:\aws\ec2 folder.
4. Create another key pair for the second instance (e.g. "Pegasus").

5. Create a Security Group

A Security Group is essentially a set of firewall rules that can be applied to your instances. A cloud-based virtual server is still vulnerable to hackers just like any other server connected to the Internet. You need to remain vigilant and follow cloud security best practices. (See the Cloud Security Alliance website for more information.)

1. Click "Security Groups" from the left menu, then click "Create Security Group".
2. Enter a name for this security group (e.g. "Stargate") then click "Create".
3. In the "Allowed Connections" section, select "SSH" then click "Save".
4. In the same section, select "HTTP" then click "Save".
5. Add more firewall rules as required. All other network traffic will be blocked.

6. Purchase a Reserved Instance

A Reserved Instance is a prepaid instance that offers "significantly" discounted usage fees compared to an On-Demand instance. In this example, a "Small" Linux/Unix reserved instance will cost $350 for a 3-yr term and $0.03/hr (roughly $22/month) usage fees, while the same On-Demand instance will cost $0.085/hr in usage fees (as of Nov 1, 2009), or about $62/month if used 24/7. Usage fees and discounts are calculated automatically. For example, if you have X reserved instances, then the first X instances currently running will be charged at the discounted usage fee, while the remaining instances are charged at the on-demand usage fee.

1. Click "Instances" on the left menu.
2. Click on the "Reserved Instances" dropdown list and select "Purchase Reserved Instances".
3. Select "Linux/UNIX" platform, "m1.small" instance type, "Best Available" zone, a term of "3 years", and "1" instance.
4. Click "Continue" then click "Place Order" to confirm.
5. You may need to wait a few minutes until the payment is processed and the reserved instance is activated. Click "Close".

7. Launch AMI Instances

An Amazon Machine Image (AMI) is essentially a disk image containing a pre-configured operating system and software. You have the ability to create your own AMI disk image from an existing server or select a pre-built AMI from Amazon’s list.

1. Click "Instances" on the left menu, then click "Launch Instance".
2. Click "Community AMIs" tab.
3. In the textbox to the right of "All Images" and "All Platforms", enter "ubuntu-images-us" for a list of official Ubuntu releases (currently Ubuntu 9.10 Karmic as of Oct 29, 2009).
4. Select an AMI from the list. Amazon charges less for 32-bit instances, so select the cheaper 32-bit "i386" AMI rather than the 64-bit "amd64" AMI. In this example, select the “ami-1515f67c” AMI ID and click "Select".
5. On the next page, enter "1" instance, “Small” (m1.small) instance type, "Galactica" key pair, and "Stargate" security group, then click "Launch".
6. Repeat these steps to launch another instance for the "Pegasus" key pair.

8. Create Elastic IP address

As mentioned previously, an Elastic IP address is simply an Internet-routable static IP address that is allocated to your AWS EC2 account and can be mapped (associated) with any running instance.

1. Click "Elastic IPs" on the left menu.
2. Click "Allocate New Address", then click "Yes, Allocate" to confirm.
3. Check the box next to the IP address, then click "Associate".
4. Select the appropriate Instance ID, then click "Associate".
5. You can now configure your DNS to map your existing domain name to this Elastic IP address and access this instance over the Internet.

9. Install and configure PuTTY

PuTTY is a free SSH client that enables you to configure and install packages on your Amazon EC2 instance from Windows.

1. Download PuTTY, PuTTYgen, and PSCP from:http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
2. Create a folder “C:\putty” and move these files to that folder.
3. Go to Control Panel, System, Advanced system settings, Environment Variables, Add “C:\putty” to the PATH system variable.
4. Run the “puttygen.exe” application from Windows.
5. Click the “File” menu, select “Load private key”, then select C:\aws\ec2\Galactica.pem (from Step 4.iii.)
6. The application will prompt you to save the file in PuTTY’s own format and click “OK”.
8. Click the “File” menu, click “Save private key”. Click “Yes” to save this key without a passphrase to protect it.
9. Save the “Galactica.ppk” file in the C:\aws\ec2 folder.
10. Run the “putty.exe” application from Windows.
11. Enter the Elastic IP address in both the “Host Name” and “Saved Sessions” text boxes.
12. On the left pane, click “SSH” then click “Auth”.
13. Click “Browse” then select the file C:\aws\ec2\Galactica.ppk.
14. On the left pane, click “Connection” then click “Data”.
15. Enter “ubuntu” in the “Auto-login username” text box.
16. Click “Session” at the top of the left pane, then click “Save”.
17. Click “Open” to verify that you can connect to this EC2 instance. Click “Yes” if the PuTTY Security Alert pops up.
18. You are authenticated via username and private key, so you do not need to enter a login password.

10. Install and configure FileZilla

FileZilla is a free SFTP client that allows you to transfer files to/from your Amazon EC2 instance.

1. Download and install FileZilla Client from http://filezilla-project.org/
2. Run the FileZilla application in Windows.
3. Select Edit, Settings from the menu.
4. On the left pane, click “SFTP” then click the “Add keyfile” button.
5. Select the C:\aws\ec2\Galactica.ppk private key file then click “Open”.
6. Select File, Site Manager from the menu.
7. Click “New Site” and enter your elastic IP address in the “Host” text box.
8. In Servertype, select “SFTP – SSH File Transfer Protocol”.
9. Leave the Logontype as “Normal” and enter “root” in the User text box.
10. Click “Connect” to verify SFTP connectivity to your EC2 instance.

11. Install the Amazon EC2 API tools on your Windows client

You will need to download your Amazon private key and cert in order to install the Amazon EC2 API tools, which are used to access additional EC2 functions (via command line or programmaticaly via the EC2 API) that are not available on the AWS Management Console web interface.

1. On your Windows machine, create the folders “C:\aws” and “C:\aws\ec2″.
2. Go to https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&action=access-key
3. Click the "X.509 Certificates" tab.
4. Click "Create a New Certificate".
5. Click "Download Private Key File" and save the pk*.pem file to C:\aws\ec2.
6. Click "Download X.509 Certificate" and save the cert*.pem file to C:\aws\ec2.
7. In Windows Explorer, make a copy of the pk*.pem file and name it “ec2-pk.pem”.
8. Also make a copy of the cert*.pem file and name it “ec2-cert.pem”.
9. Download the EC2 API Tools from http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351&categoryID=88
10. Unzip the files into the “C:\aws\ec2″ folder.
11. In Windows, click the "Start" menu, "Settings", “Control Panel”, “System”, click “Advanced system settings”, then click “Environment Variables…”.
12. In the bottom window called “System variables”, click “New…”.
13. Enter “EC2_PRIVATE_KEY” as the Variable name, enter “c:\aws\ec2\ec2-pk.pem” as the Variable value, then click “OK”.
14. Click “New…” again in the bottom window called “System variables”.
15. Enter “EC2_CERT” as the Variable name, enter “c:\aws\ec2\ec2-cert.pem” as the Variable value, then click “OK”.
17. In the bottom window "System variables", find and select the variable named "Path".
    Click "Edit…", insert "%EC2_HOME%\bin;" at the beginning of the Variable value, and click “OK”.
18. In Windows, click the "Start" menu, click "Run", enter "cmd", and click “OK” to open a command prompt.
19. Enter the following command to verify that the EC2 API command-line tools are working properly:
C:\ec2-describe-regions

12. Install software on your Amazon EC2 instance

• You can now install software packages on your Amazon EC2 instance. Keep in mind that although this is a virtual server, you still need to follow the license agreements that come with your software.

Optional: Terminate the Instances

• If you are just following along and no longer need these instances, you can terminate them to avoid further usage charges. However, note that all your changes will be lost if you have not yet bundled an AMI snapshot of this instance (described in Part 1.2: AWS Data).

1. Click "Instances" on the left menu.
2. Click the checkbox of the instance you want to terminate.
3. Click "Instance Actions" dropdown list and select "Terminate" to shutdown this instance.

V. Next Steps

• In Part 1.2: AWS Data, I will explore the use of Amazon S3 and EBS for cloud-based storage and Amazon SimpleDB and RDS for database applications.
• In Part 1.3: AWS Fault Tolerance, I will explore the AutoScaling and Elastic Load Balancing features of Amazon Web Services.
• Please see The Mobile Cloud Project page for more info on the other parts of this project.

VI. References

1. Amazon Web Services Technical Documentation
http://aws.amazon.com/documentation

2. Cloud Security Alliance
http://www.cloudsecurityalliance.org

3. Ubuntu Server Edition on Amazon EC2
http://www.ubuntu.com/products/whatisubuntu/serveredition/features/ec2