Few days ago, I was still struggling with solution to further scale this blog so it will serve contents faster and at the same time, not pressuring the server too hard. Hongkiat.com serves about 50,000 pageviews daily and that consumed about 60-80Gb of bandwidths on a daily basis. Something have to be done here so the requesting of images and files will not affect the stability of the entire server. After some readings, considerations and research, I settle for Amazon S3.

You might have heard of it, or perhaps using it already. But for those who have problems scaling your site, looking for solutions or looking for a stable online file hosting, here I’ve written a fairly complete article (I hoped) that give you a basic understanding on Amazon S3, together with guides on getting and account to using it. For the ease of reading, contents are spitted up in the following sections.

• In a Nutshell
• Gettting an Amazon S3 Account
• Using Amazon S3
• Amazon S3 as Image Hosting
• Amazon S3 Applications and Other Resources

Full guide after jump.

In a Nutshell

Amazon Simple Storage Service, also known as Amazon S3 is an online storage facility. It is cheap, fast and easy to setup. And since it’s a service provided by e-commerce giant Amazon, you can be rest-assured whatever you stored at S3 is secured. Read more about Amazon S3.

Who needs Amazon S3?

In S3, there’s no initial charges, zero setup cost. You only pay for what you utilize. It is utmost suitable for webmasters and bloggers, especially those who have the following issues:

• Running out of bandwidths

  If you are on shared hosting account, any Stumble Upon or Digg effect can easily eat up the entire bandwidth limit for the month. Most of the time, the web host will suspend the account until you have settle the payment for the extra bandwidths consumed. Amazon S3 provides unlimited bandwidth and you’ll be served with any amount of bandwidth your site needs. Charges will be made to credit card and payment can be made at the end of the month.

• Better scalability

  Amazon S3 using cloud hosting and image serving is relatively fast. Separating them away from normal HTTP request will definitely ease the server load and thus, guarantees better stability.

• Paying for more that you actually used

  Whether you are on shared hosting, VPS or dedicated server, you pay a lump sum each month (or year) and the amount includes hard disk storage and bandwidth you might not fully make use of. Why pay for more when you can pay only for what you are used.

• Store files online

  Instead of backing up your files in CD/DVDs to save more hard disk space, here’s another option. Store them online, and you have the option to keep them private or make them public accessible. It’s entire up to you.

• Easier files retrieval and sharing

  If you store your file online, you can access them anywhere as long as there’s Internet connection. Amazon S3 also allows me to communicate files better with friends, clients, and blog readers.

Unlimited storage and bandwidths, pay as you use, full control on file privacy are what excites me towards migrating images on hongkiat.com to Amazon S3. You can probably think of more that suites your need. Read more on Why you should use Amazon S3.

Next, I’m going to explain on how you can sign up for an Amazon S3 account.

Gettting an Amazon S3 Account

Before we go into signing up an account, I think you should at least know how Amazon S3 charges. Check them out over here, or estimate with a AWS Simple Monthly Calculator. Now if you’re all set, let’s get an Amazon S3 account.

1. Sign-up/Login to Amazon

   If you have an Amazon account, login, else sign-up for one.

   

2. Get Amazon AWS Account

   Go to aws.amazon.com and sign-up a Amazon Web Services Account.

   

3. Look for – Amazon Simple Storage Service (S3)

   Once you are done signing up, you’ll be greeted with a page that says your account has been created and information has been sent to your email. Look for Amazon Simple Storage Service under the list, click it.

   

4. Sign up – Amazon Simple Storage Service (S3)

   Once again, you’ll be brought to Amazon S3 introduction page. Read it again if you need, or just skip to signing up an account by clicking on the Sign up For This Web Service. Put in your credit card details and follow the instruction to setup your account.

   

5. Know Your Username/Password

   Once you’ve successfully sign-up, Amazon will prompt you on your AWS Access Identifiers, which includes your Access Key ID and Secret Access Key. Note that Access Key ID and Secret Access Key are as good as your username and password so you should keep them safe.

   

   If you have missed theAccess Key ID and Secret Access Key notification, click on Your Web Service Account, choose AWS Access Identifiers to retrieve them.

   

Under Your Web Services Account is also where you check the account activities, how much you are going to pay at the end of the month, changing your profile etc. Getting familiar with these pages is necessary.

Now your Amazon S3 account is created and ready to go. Let’s do some uploading.

Using Amazon S3

Your Amazon S3 account starts with a clean root account. On the root is where you create buckets. Bucket is Amazon S3’s terminology for root folder. You can create multiple buckets, and inside buckets is where you place your folders and images.

Amazon S3 releases a set of API and developers around the world releases application that allows your Amazon S3 account to talk to your local computer so you can do all the file uploading, synchronization, back-up , etc. For starters, we’ll be looking at how you can take advantage of S3Fox extension from RJonna (Firefox extension) to connect to your Amazon S3 account and later, we’ll give you a list of alternatives of free and paid applications to connect to Amazon S3.

Using Amazon S3 with Firefox S3Fox

S3Fox is a Firefox plugin, so if you don’t have a Firefox browser installed in your machine, you’ll need to get one. Install S3Fox plugin, have your Access Key ID and Secret Access Key ready, let’s get started.

1. Launch S3 Organizer

   In Firefox, go to Tools, select S3 Organizer.

   

2. Set up account

   Set up your Amazon S3 account with S3 Organizer. Enter
   a self explanatory Account Name, your Access Key and Secret Key. Click Add.

   

3. Get connected, create first bucket

   Once you’ve entered the correct information, you’ll be brought to your account (which is blank, by default). On the left side of S3 Organizer will be your local machine folders, and Amazon S3 on the right.

   Right-click, Create Directory. Anything created on root level will be your buckets. All files and folders will be stored/organized under buckets.

   

4. Create folders, upload images

   Double click into your bucket, create a folder. Inside the folder, upload an image. By default, anything uploaded to your Amazon S3 account will not be accessible by public.

Amazon S3 as Image Hosting

By default, images uploaded to Amazon S3 with S3 Organizers will not be made public. If you intend to share uploaded files with your friends and peers, or if you want to use Amazon S3 to host your website’s images, additional steps will be needed.

1. Edit image permission

   Right-click on one of the image uploaded, select Edit ACL.

   

2. Make public accessible

   To make your image public accessible, make sure
   Everyone, Authenticated Users and me(Owner) has read access.
   Follow the settings in the image below. Click on the icon to swap between ticks and crosses.

   

3. Get image URL

   Right-click on any particular image, select Copy URL to Clipboard. Your URL will look something like this:
   http://hongkiat.s3.amazonaws.com/10yearsago/amazon_10ya.png

   

   Image URL comes in the following fixed format:
   http://bucket_name.s3.amazonaws.com/foldername/filename.jpg

Amazon S3 Applications and Other Resources

We’ve been using S3Fox throughout the entire explanation because it’s free and it resides on Firefox browser. But I thought you should also be aware of other applications and various ways out there that provide similar facilities.

Amazon S3-Supported Applications

• JungleDisk – Reliable online storage powered by Amazon S3.

  

• Transmit – FTP/SFTP application for Mac.

  

• S3Sync – Consist of S3syncs and S3cmds. Ruby program that allows control of Amazon S3 account with shell commands.
• Bucket Explorer – User Interfaces for Amazon S3.
• Backup Manager – Command-line tool for Linux.
• S3 Backup – Windows desktop application that makes it trivial for everyone to use Amazon’s impressive infrastructure for remote backups and secure online file storage.
• jets3t – Toolkit for Amazon’s S3 online storage service.
• Sync2S3 – Synchronizes your files with the Amazon (S3), providing you with a secure and affordable backup solution.
• SME Storage – Access files from anywhere.

More Online References

Here’s more online references to help you understand Amazon S3 and its connectivity better.

• Amazon Simple Storage Service – Browse through the solutions that Amazon Web Services developers have built using Amazon Simple Storage Service (Amazon S3).
• Amazon S3 Forum – S3 Forum with plenty of useful information.
• Amazon S3 API – Understand Amazon S3’s API.
• Amazon S3 in Wiki
• Using Amazon S3 as an Image Hosting Service
• How I Use Amazon S3- Read Paul’s method to use Amazon S3 with rsync and JungleDisk.
• Bulletproof Server Backups with Amazon S3 – Control S3 with Ruby and S3Sync.
• How to Use JungleDisk, Amazon S3, and rsync to Backup Your OS X Home Directory
• Cheap Server Backup with Amazon S3
• Amazon s3 backup for web server files and SQL using bash
• Using Amazon S3 from Perl
• Using Amazon S3 with Django

Did I leave out any good resources?