Category: AWS
May 4, 2007
Amazon Web Services Pricing Change
Amazon has recently released that their prices for their web services are going to be changing on June 1. Is it cheaper?
For Amazon EC2 (Elastic Compute Cloud), it definitely is. The AMI instance rates have stayed the same at 10 cents per hour, rounded up. However, bandwidth fees have dropped and are now on a tiered basis. The bandwidth costs are the same across all of the services and bandwidth between EC2 and S3 (Simple Storage Service) is still free.
For Amazon SQS (Simple Queue Service), the costs have remained basically the same, with the exception of the bandwidth changes. This should also cause, if anything, a reduction in fees for SQS users, although not by much.
However, for Amazon S3, the cost structure has changed. No longer is there just storage and bandwidth costs. There are now separate fees for PUT and LIST requests and another separate fee for GET requests. The storage pricing has also remained the same at 10 cents per month per gigabyte. However, now, each 1,000 PUT or LIST requests will cost an additional penny and every 10,000 GET requests will cost a penny. That doesn't sound like much, but for very high volume but very small data, the costs could go up. They also specifically do not say that this is free for EC2 users. The impact of that is that trying to use S3 as a database of sorts for small bits of data could get much more expensive.
Amazon says this about the price changes to S3:
To quantify the impact of these pricing changes, we looked at what the effect would have been on customers' March 2007 bills. Assuming this change was in effect, 75% of customers would have seen their bill decrease, while an additional 11% would have seen an increase of less than 10%. Only 14% of customers would have experienced an increase of greater than 10%.
So that does mean one in every four users of S3 will see a rate increase. Now, that could also include users that are at the very low end. Maybe some folks spend 25 cents a month and might now spend 30 cents a month because of rounding for transactions fees. That's more than 10 percent, but it's all still pennies.
The bandwidth fees are were most of the costs have changed. Upload and download bandwidth is now charged at a different rate. Upload has been reduced to 10 cents per gigabyte, half of the original 20 cents per gigabyte per month. This makes using S3 to back up data much cheaper. It also reduces the cost of uploading new AMIs for EC2. Download bandwidth is now tiered. For bandwidth used up to 10 terabytes per month, the cost is now 18 cents per gigabyte, down 10%. The next 40 terabytes of data transferred will cost 16 cents per gigabyte, down 20%. After that, bandwidth after the first 50 terabytes in a month is charged at a mere 13 cents per gigabyte per month. That's a hefty 35% reduction.
So, a while back I looked at the cost of Amazon EC2 against a dedicated host. In that writing, I had no examples of bandwidth in the double-digit terabyte range. I was looking at it more from a personal point of view. There, we saw that an AMI that used 40 gigabytes of storage and 100 gigabytes per month of transfer would cost about $99 per month. Now, this cost would be reduced to $97 because the bandwidth would drop from $20 a month to $18 a month. The initial upload for the 40 gigabytes would drop from $8 to only $4, as well.
Now, if you're a large user or have a very popular podcast and deal in the terabyte cost range, things may get much better. If you were using a dedicated host at $119 like I am and had to pay 50 cents per gigabyte for transfers after the first 1500 gigabytes in a month, you'd spend about $35,119 a month for the hosting to use 71,500 gigabytes a month in bandwidth. At a sustained rate of 222 Mbps, though, a standard 100 Mbps server won't be able to keep up. Spread across 12 servers to have each at 20% of the Ethernet speeds, spread evenly 24 hours a day, you'd add another $1,400 to the cost each month and server complexity.
If, however, you hosted via Amazon EC2 with the files directly available off of S3, you'd likely only need a single EC2 to serve up the feeds. To be safe, though, you'll use two of them at a cost of $146 per month. Your first 10 TB of bandwidth will cost $1843. Your next 40 TB will add $6553 per month. The last 21.5 TB will cost an additional $2,862. So far, were up to a total cost of $11,404 per month, or nearly $25k cheaper, so far. Now, don't forget about the S3 GET fees. Every 10k downloads costs an additional penny. Assuming your average data file size was 20 megabytes (a podcast, remember?), using 71.5 TB would be about 3,750,000 GET requests. That adds all of $3.74 a month. Were you worried? If you hosted your feed on S3, too, it might have 100 times as many requests as RSS readers check it every hour or less. That could bring the costs up to hundreds of dollars. But, as you can see, it would be cheaper to host that on the EC2 directly.
So, not only is AWS cheaper, but it's also much more flexible. You could increase the bandwidth by 10x without having to add your own servers because the S3 system will just handle it (presumably). That's very convenient.
()
(0)March 8, 2007
ElasticLive: A Comparison to Native EC2
So, there's this service out there called ElasticLive, run by enomaly. At first, it looked like competition to Amazon EC2, but then I saw how they were wording their services and it didn't take long to realize that they are reselling Amazon EC2 services. Amazon EC2 is already rather easy to use, so I am curious as to what they offer over Amazon EC2.
For starters, for their dedicated grid machine they charge 25 cents per hour, 45 cents per gigabyte of bandwidth, and 25 cents per gigabyte a month for storage. However, they include 1000 GB of bandwidth and 160GB of storage in this pricing. Let's compare, shall we?
For someone maximizing the account, ElasticLive will be losing about $115 a month. Even with 160GB of storage, bandwidth only has to average below about 425GB, though, and they make money. Then, again, above about 1250GB of bandwidth they'll start making money again. So, for the consumer, it's only cheaper if you're in that middle-ground.
However, they offer fractional virtual servers (likely a VM system within a VM). For 5 cents an hour, you get a system with 20GB storage, 200MB RAM, and 60GB of bandwidth for the month. Given the normal system has 1.75GB of RAM and 160GB of space, this gives about 8 virtual machines per, uh, virtual machine. Let's compare this, shall we?
- A full Amazon EC2 native AMI with same transfer and storage: $88.04
- 1 ElasticLive fractional: $36.52
- 8 ElasticLive fractional: $292.19
- Amazon EC2 with 8x transfer and 8x storage: $193.04
Here, we can see the consumer that just needs something small and simple can save some money, especially if they don't need much space or bandwidth. They can get many of the Amazon EC2 advantages but at a lower cost because ElasticLive is helping divide the resources out. Assuming they can get 8 instances on one AMI, they'd be making money with only 5 or 6 fully used fractional grid machines.
It would also seem that ElasticLive offers some basic AMIs that can be used as part of their service, including access to server configuration tools like cPanel and Plesk. It's an interesting idea, for sure. However, it seems to have a fairly narrow window of use to the consumer, unless their provisioning tools are very easy to use compared to something like the EC2 UI plugin for Firefox. Although it's a common thing to do, I'm not sure about the prospects of building a business off of the fact the most consumers don't fully use what they pay for. I'd also be curious if someone could access their personal AMIs in their own AWS account for the fractional stuff or not.
One service that they do offer that is interesting is the ElasticDNS service for dynamic DNS for Amazon EC2 users. I'll have to check that out a bit more...
March 6, 2007
The Cost of Amazon EC2 P2V of an Existing Server
So, let's say that you have an existing server that you'd like to migrate to Amazon's Elastic Computing Cloud (EC2) by using their physical to virtual (P2V) tool. The question is, how much will this cost you for the initial Amazon Machine Image (AMI) upload to Amazon Simple Storage Server (S3)? And then, how much will it cost to continue to run on EC2?
The basic equation is relatively simple. We'll go with an example, first. Let's say you've got a server that's currently using 40GB of storage. That will need to be packaged into an AMI. Although there may be some compression (?) we'll assuming there isn't. Uploading 40GB to S3 will cost 20 cents a gigabyte, or eight dollars (the costs are all in US dollars). After that, you'll be paying 15 cents a gigabyte per month to store this AMI or $6 per month plus bandwidth. If you were using about 100GB a month of bandwidth, you'll be paying an addition $20 a month for that. That's a total of $26 per month plus the cost of the instance at about $73 per month. So, you'd then be paying about $99 per month.
However, if you just want to bring up your server in the cloud for a few hours to run some tests or experiments with different configurations, you'd only pay $8 to get it up there and then for the AMI time.
There is one caveat, though. Make sure that you aren't going to be charged by your current host for the upload. Also, you need enough room on your current physical machine to make a mirror of it.
In my case, I'm thinking about splitting a single physical server into two servers. One that will remain the physical server and one that will be hosted with EC2. This would increase costs, of course, but it creates a better logical separation between certain uses of the machine.
My other use case is to test a yum-based OS upgrade since I can't really upgrade any other way - aside from imaging a slightly newer OS on to the box and reconfiguring everything. However, I noticed the following in one of the FAQs:
Can I use my own kernel?
Not at present.
I'm not entirely sure what that means given that people have published many AMIs with many different version of Linux which I didn't think would be using the exact same kernel.
Finally, along the cost lines, don't forget about your time. That has value to you and/or your company. To summarize the formula, you've got:
One Time Cost:
(20 cents * GBServer) + (current host cost to upload) + (your time to prep upload) + (your time to convert to S3 store for persistence)
Monthly Costs:
($73 per server) + (15 cents * GBServer) + (20 cents * bandwidthGB)
Sure, most of this is obvious. The largest unknown, though, would be the time to convert servers over to have data persist outside of the AMI. This can either be done by having the application directly store to S3, having a backup system run a frequent full backup of the data directories to AMI and restore on boot, or use one of the filesystem drivers to have the data portion of the filesystem be linked directly to S3. (Where S3 is used, any other off-AMI storage could be considered, but I'm assuming the network connection between the AMIs and S3 is very fast and very low latency.)
March 1, 2007
Useful Amazon S3 and EC2 Tools
So, in the last couple of weeks I've found a few tools that have made using both Amazon S3 and Amazon EC2 a little bit easier.
First, is Jungle Disk, which is less of a tool and more of a way to easily leverage Amazon S3 for storage. This is an application that when run on Windows allows you to have driver letter access to Amazon S3 storage. It uses it's own bucket so you can't browse and copy just anything, but the ease of using it is amazing. It also runs on Mac OSX and Linux. It includes full encryption, too.
Second is NS3 Manager, which is a graphical browser for your Amazon S3 account. It also allows for copying of objects to and from your file system. In this case, any object is support but it doesn't provide drive letter access; you have to go find the object and choose transfer. It does make creating bucks easy and copying files to them easy, though. You can then modify the ACLs, too, without having to resort to API calls. One feature that it could use is a URL generator to easily get the URL to a public object you own. They are easy enough to figure out, but it's nice to be able to copy'n'paste stuff, y'know?
Finally, we've got a Firefox plugin called EC2 UI that provides an interface for seeing AMIs you have available, starting them, and terminating them. In addition, it provides an interface for managing firewall groups and key pairs. This does a lot of what the command line tools do, but with a nice UI to it all while running in your favorite web browser. ;)
I'm sure I'll be using more as time goes by, but these seemed to be the easiest to get, understand, and use. Of course, when I get around to doing anything interesting I'll likely be using the APIs directly.
February 26, 2007
Thoughts on Amazon EC2
So, I finally got a chance to play with Amazon EC2, or Amazon's Elastic Computing Cloud. The experience largely went how I thought it would. However, there were a couple of interesting things that I noticed. These things revolved around security, the virtual machine environment, and the billing of usage.
First off, EC2 has a firewall around it. This is controlled via an API that allows you to authorize or revoke privileges based on many different rules all combined in to named rule sets. The rules are applied immediately, too, which is great. Compared to many normal data centers where the firewall might not even be accessible to you for your dedicated servers, this is particularly useful. Not only can you defined traffic from IP and IP ranges to specific ports but you can also defined traffic by security group and user. So, for instance, you could have a backend machine in the cloud that an only be accessed by a web server group you defined. Or, instead of that, you can just specific your username and all instances running as your username will have access to that machine. This is fast, flexible, powerful, and very easy to maintain. For higher security, other software firewalls can be put in place and you could even dedicate an entire instance to being a firewall, should you choose to.
Second, the environment that the OS is running in is basically as described. However, it's listed as being equivalent to a 1.7Ghz processor, but the procinfo shows it being an AMD processor running at 2.4Ghz. I haven't done any tests on the performance of the machine, but the response times are nice and snappy. Even if the compute power isn't fully that of the processor listed, the virtual configuration is running on top of high quality components so all parts are sufficiently fast. I'd be curious how hard to would be to create FlashMob networks with a handful of instances, although it doesn't look like much has been happening within that community of years. There really doesn't seem to be anything preventing a group from signing up to use more than 20 instances and going forward within bringing up any form of high performance computing cluster for science or profit. The ability to do so, even if it's just to test the software and proof of concept, without having to dedicate hardware is wonderful.
Finally, a couple of items about billing surprised me a little. Now, this has to all be taken into context. My little experiment ended up costing me maybe a dozen cents more than I thought it would. The billing for EC2 is listed as such: 10 cents per hour per instance being on, plus 20 cents per GB transferred in our out of Amazon (but not between S3), and 15 cents per GB per month for the storage of instances on S3 (billed by S3).
The first surprise was that the 10 cents per hour used isn't pro-rated. That is, 1 minute costs 10 cents, as does 59 minutes. So, just to boot an instance will cost you 10 cents. The impact of this is that booting 60 machines for one minute would actually cost 6 bucks not 10 cents like I may have implied before. It's still cheap, but it has implications for the above mentioned performance cluster tests; that is, assume anything you want to experiment with will cost up to an hour and thus you might as well use up to an hour. This is also true for machines that are on 61 minutes; you will pay for two hours or 20 cents. Don't get me wrong, though. This is still cheap for experimentation. It just might not be the pennies you thought to bring up a machine for 10 minutes a day.
The second surprise turns out to not be a surprise. At first, I thought that S3 was billing for the transfer in and out of S3 and that EC2 wasn't, so it wasn't double-billed. However, the original assumption that no traffic between S3 and EC2 is billed is correct. See, after creating the AMI that you'll be offered to create during the "getting started" document, I saw a charge appear on my S3 activity in the bandwidth. However, I didn't do the math on it. The charge was for a penny and I thought maybe it was for the transfer. As it turns out, it was likely just a round-up from other usage (still shows 0.000GB of bandwidth). The AMI is about 227MB, which does show up in the usage detail of the transfer (one time, though), but clearly wasn't charged for. See, if it had been for transfer like I was thinking, the fee would have been 5 cents.
Now, again, keep in mind that although the billing of the instances is rounded up to the nearest hour, we're still talking pennies. Even for non-business use, which is how I'm currently experimenting with it, this is cheap. So far, I have 20 cents in instance usage, a penny for S3 storage usage, and another penny for bandwidth usage. I've learned a lot about it without even spending a quarter.
My next tasks with Amazon EC2 are to run some performance tests to try to determine what kind of compute performance is really meant by 1.7Ghz. That is, is it like a 1.7Ghz Core 2 (solo, of course) or like a 1.7Ghz Pentium 4? There's a huge difference between the two and it might actually be more like a 2.4Ghz modern AMD processor, which would also be great. Another task I want to do is see if I can turn my current server into an instance. I've been a little confused by some of the AMI creation documents without actually doing them, so I've just got to roll forward with it. That's a topic for another post, though.
Until next time, compute on!
