ec2

I run quite a few EC2 instances on Amazon's cloud infrastructure, mostly running Ubuntu. 99% of the time, everything works great. But every so often one of my instances will blow up for no apparent reason. Usually this is preceded by a spike in disk IO.

So first things first, I try to connect with SSH to no avail. Then I login to AWS' EC2 Management Console and check the instance. Sure enough, one of the status checks has failed and it says that the instance is unreachable. I check the system log and everything looks fine. So according to the EC2 documentation, the options to fix your failed instance are as follows:

• Reboot your instance
• Change your operating system configuration
• Start a new instance

My favourite suggestion is the second point because, you know, the instance is readily available to "change my system configuration".

Anyways, rebooting the instance never works. And I mean never. Changing the system configuration is not an option because the system is not available and the last point is a pain in the ass because it involves a bunch of work in remounting volumes and changing configuration, moving things around, etc.

What I have found though is that actually stopping and starting the instance (assuming its an EBS backed instance) will usually fix the problem. So I stop the instance, start it, associate any elastic IP as necessary and voila, magically fixed. Hopefully this helps someone out there because its not mentioned in the AWS EC2 documentation.

I recently contracted out some work for a project I am working on in Drupal because it involved CSS, HTML and theming and quite frankly, my skills in that area suck. I used ODesk because I do some other data entry outsourcing there and it works quite well. Anyways, it was relatively easy to find a qualified person to do the work. Now the supposedly hard part, setting up a sandbox for a remote worker to access and do the work. This was a rush job that needed to get done yesterday so I needed something setup quickly.

Well luckily with Amazon EC2 and its nifty cloud, this turned out to be quite easy. I created a a micro instance in the US East (Virginia) region which costs a whopping 2 cents per hour (works out to about $15 / month). This can be decreased significantly by purchasing reserved instances. I used an EBS-backed instance so that I can start / stop it at will and not have it running if I don't need. I also assign a custom security group to it so I can add or drop custom firewall rules and it doesn't affect my other live instances. Finally, I use Ubuntu 10.04 LTS and Alestic's AMIs because its fast and simple to setup and run.

So now I have a running Ubuntu instance, now what? Well with a few simple commands, I can get a Drupal website up and running. Keep in mind this is a short term server that I need to focus on ease of use, not security so what I am doing would not be appropriate for anything else.

Copy authorized keys to root .ssh dir so I can login as root:
sudo cp ~/.ssh/authorized_keys /root/.ssh/

Now logout and SSH back in as root.

Update your Ubuntu instance:
apt-get update && apt-get -y upgrade

Install LAMP server:
tasksel install lamp-server

Edit your default site Apache configuration file with your favourite editor and point to your new htdocs dir (/home/htdocs):
pico /etc/apache2/sites-available/default

Enable mod rewrite for Drupal and reload Apache:
a2enmod rewrite
/etc/init.d/apache2 reload

Login to MySQL and create your database:
mysql -p
create database databasename

Now ssh to your live/development server and create a backup of your site with db to move to sandbox:
cd /home/htdocs
mysqldump -p databasename > db_date.sql
cd ..
tar .zcvf mywebsite.tar.gz htdocs/
rm htdocs/db_date.sql
scp mywebsite.tar.gz sandbox:/home/htdocs/


Now flip back to your sandbox and extract your tar file:

tar xfz mysqwebsite.tar.gz


Edit your Drupal settings file to point to database:

pico sites/default/settings.php


Load your database from the dump and delete the dump file:

mysql -p databasename < /home/htdocs/db_date.sql
rm /home/htdocs/db_date.sql


Now your website should be operational.

For quick access to your remote developer you can simply enable password security for your SSH daemon by changing PasswordAuthentication setting to Yes, assign a password for root and reloading SSH daemon:

pico /etc/ssh/sshd_config
passwd
/etc/init.d/sshd reload


Because most FTP clients support SFTP (FTP over SSH), your remote developer can now access your sandbox and you should be good to go.

Keep in mind that giving out root access is not ideal, nor is most of what I have talked about here good for security. In fact, it flies in the face of most best practice security measures that should be taken for any server. But if you need to get a server up and running fast for a remote worker to get stuff done quickly, it works and does the job.

Make sure you stop and/or terminate your instance as soon as you are done with it! Enjoy.

Its been a while since I posted anything so I thought I would talk about how I've setup my servers using Amazon's EC2 cloud services for easy administration. I have several instances running mostly Ubuntu 10.04.

Some of this stuff may be pretty simple to you depending on your level of expertise but it really simplified administration for me so I wanted to share.

I put all my keys in a folder in my home directory and then I have a file named config in my ~/.ssh folder that looks something like this:


Host server_alias
User ubuntu
Hostname public_address_to_server
IdentityFile ~/.keys/key_file.pem
Localforward 6033 localhost:3306


I have an entry like what is shown above for each server. Once you have a host setup in your config file, you can ssh to your server with one very simple command:

ssh server_alias

Wow, that was easy. There's also a bunch of other cool things you can do once you have this setup.

You'll notice I have a local forward setup on port 6033. This allows me to connect to my remote MySQL server (running on its default port of 3306) once I have established an SSH connection using the port 6033. I can use MySQL GUI tools or simply connect with the MySQL command line utility like so:

mysql -h 127.0.0.1 -P 6033 -p

I am not sure why but for whatever reason, it doesn't work if I enter localhost but if I use 127.0.0.1 it does. Weird.

File sharing over ssh is also super simple. Simply use the sshfs command line tool:

sshfs server_alias:/home/whatever_dir ~/server_alias

Booyah, now I can manipulate folders and files on my remote server locally. If you want to kill the share, simply use:

fusermount -uz server_alias

Props to @netwonk for helping me get a lot of this stuff going. Happy server administrating!

Amazon recently announced that they are giving away a free micro instance to new accounts on their EC2 service. They call it their free usage tier.

So what is a micro instance then? Well, its their smallest computing unit in Amazon's cloud service. Basically, fire up one of these bad boys and you have your own little virtual server that exists somewhere in Amazon's vast computing nebulous.

What can you do it with it? Anything you want really. But most people will probably run linux on it and set up a LAMP (linux, apache, mysql, php) server to run web sites / applications.

What kind of experience do I need? Unfortunately, you need some skills to use Amazon's cloud services, its like running your own server with a little added complication.

I have found that micro instances are decent for most small to medium websites and testing. However, they can be a little underpowered especially with memory because they only have 630 something megabytes of RAM.

But who can complain with free? Existing customers like me I guess who the deal doesn't extend to.

More information on Amazon's EC2 free usage tier.