If you have ever done any searching for a web hosting company for your site, you have undoubtedly come across web hosting companies promising "unlimited hosting" for some "to good to be true" price. You asked yourself "how is this even possible" when you see other hosting companies offering the same package for much higher prices. So, is unlimited hosting really true? As far as I know, the only thing that is truly unlimited is outer space. That being the case, how can these hosting companies offer "Unlimited Hosting".

First, you must consider that their traditional customer, or the customer that they are targeting will not use very much in the way of resources. The typical web site will use minimal bandwidth or traffic, minimal disk space and may have only one or two email addresses. Of course, this is what the host is counting on and this offering is a great way for them to get your business. However, should you start using what they consider too much bandwidth or disk space, you will find a very nice email in your email box telling you that you're going to have to upgrade some part of your plan because "we found that you are utilizing resources to the point that is affecting other customers" or some other similarly worded email. The key to unlimited hosting is not only knowing, but also understanding, the fine print.

If you read the fine print at many of these hosts (which may not be so easy to find) you will see numerous caveats. Below are just a few of them I've found during my research.

1.We reserve the right to change the terms of the package at any given time. They usually list some time frame for the notice - usually 21 to 30 days.

2. Email accounts have limited storage capacity. Of course, for a small upgrade fee you may increase your email storage.

 

3. Backups are not included but for a small fee you can add a backup plan

4. Your bandwidth is part of a shared network. This means that they have allocated a pool of bandwidth for the unlimited customers. The downside to this is that any customer could effectively cause your site to run slower due to the lack of unavailable bandwidth.

5. Large videos are not allowed. What they consider a large video is anybody's guess.

6. Support is not included with this plan.

7. You may not install any scripts which may affect the performance of the server.

8. You can add all the content you wish but maybe not all at the same time. The vast majority of our customer's sites grow at rates well within our rules, however, and will not be impacted by this constraint. What exactly does this mean?

9. You may not use your disk space as an off-site backup source.

10. Database servers have a limit to the number of concurrent connections.

Does this mean that you should never use these hosting companies? Of course not - but you should definitely approach with caution and keep expectations low. If you have a small site that you know won't use a lot of disk space or server resources, then a small affordable "unlimited hosting" package may be perfect for you. Just remember the phrase "Caveat Emptor" which is Latin for "Let the buyer beware".

What exactly is a CDN?

As someone who runs a hosting company, I get this question on a regular basis as we have a couple of different CDN offerings. Many people have heard of a CDN, usually from a friend or an online article, but few people know what it does or how it works, so let's start with a definition and overview.

CDN is short for Content Delivery Network. CDNs were created to deal with proximity issues when delivering content from a web site. What this means is that the further away the surfer is from your web site, the slower it will load for them. If your web site is hosted in the US, a surfer in the US is going to see much faster load times than a surfer in Australia, simply because they are closer to the web site server.

A CDN consists of storage servers called "caching servers" or "edge servers" that are strategically placed around the world. Web site content is pulled from the "origin server" (the server that hosts your web site) and is pushed to all the edge servers. When a surfer makes a request for that content, the CDN first determines where the surfer is, then finds the nearest edge server to the surfer and finally confirms that the requested content is on the edge server. If the content exists on that edge server, it delivers that content from the edge server to the surfer, thus providing faster delivery of the CDN content to the surfer. If the CDN determines that the content should exist on the edge server but for some reason does not exist, it will pull it from the origin server and place it on the edge server. By using href tags in the HTML code for the site, the webmaster can control what content should be on the CDN and what content should be delivered directly from the origin server. Now that we know what a CDN is, let's determine if you need one.

In the description above, we learn that a CDN has to go through a decision process to determine what to do (where is the surfer, does the content exist, etc). This decision process takes a second or two, which can add a delay to the loading of the content. Due to this decision process, it should be apparent that a CDN is not ideal for all sites or all content. A CDN was designed with larger content (files) where a 1 second delay is not critical. A CDN is also designed for "popular content" i.e., content that is accessed often. Edge servers do not have infinite disk space so all CDNs automatically expire or delete content on the edge servers if the content has not been accessed in some time. This time span is configurable by the site owner but usually it's 30 days. So, it would not make sense to put up content that is only accessed occasionally.

A typical web site is hosted in a single location at a hosting company, and the web site usually consists of web pages of text and images and has been, or SHOULD BE optimized for the best performance. If a web site is properly optimized, it should load pretty quickly from anywhere in the world. Text is very small and most web site images are very small as well. Putting this small content on a CDN will actually defeat the purpose due to the decision process explained above. However, if you have large images, large downloads (zip files, software files, etc) or movie files, a CDN is perfect for this application, provided you have popular content to deliver. We have found that the best use of a CDN is to deliver the web site text and smaller images directly from the origin server and put the larger files on the CDN, thus providing a "best of both worlds" experience for the surfer. Large streaming movies will play much faster on the CDN. Large zip file downloads will download much more quickly on the CDN. The one exception to putting smaller files on the CDN would be a JavaScript file, or some other file that never changes and is always loaded on every page. We have seen some customers use the CDN for JavaScript files and have good success with them.

What a CDN is NOT good for...

CDN edge servers are just very large disk arrays and they are very good at delivering content; however, they are not designed for processing, so you usually cannot put php files or any other scripts/programs that require server side processing. You can put javascript on them because it is processed on the client side by the browser. CDN providers want their edge servers to "shovel content" to the surfer and nothing more. Adding in server side processing would simply slow down the CDN and also create a new level of un-needed complexity.

CDNs are also not cheap. Because of the infrastructure required, as well as the software that runs it, there is a significant investment required to build a CDN. This means that CDN bandwidth can run 2-10 times more than regular bandwidth provided by your hosting company.

Finally, if most of your surfers are in the same area as your web site, i.e., your web site is in the US and most of your surfers are in the US, there is no benefit to having a CDN, as you'll be paying extra but not really seeing any faster speeds.

So, if you have a web site that delivers larger files, whether streamed or downloaded, you have surfers all over the world and you wish to give them the best possible experience, it may be time to see if YOU need a CDN.

In my previous posting, I discussed the difference between throughput (95th percentile) and transfer (per gig). In this article, we're going to dig a bit deeper and discuss the advantages of metered (un-capped) versus un-metered (capped) bandwidth.

We start with a question from Jonas, a reader, who asks, "With 95 Percentile, can I have my transfer rate limited to some upper limit so that I have a cap on what my bandwidth cost will be each month? I would worry that if I had 2 or 3 days with 10x my normal traffic I would have a heart attack when I get the bill."

In a word: YES. You can have your bandwidth CAPPED, if you like, and if your hosting company will allow it. With that said, and I can only speak for NationalNet, we do not recommend our clients to cap their bandwidth and this is based on sound business practices. Here are some examples:

Example #1: You own a web site that earns you revenue, either by selling your wares on it or your clients pay a monthly fee to access your content. For whatever reason, your site starts getting an unusually high amount of good traffic, bandwidth goes UP, but so do REVENUES...and thus so do PROFITS! However, if your plan is capped, once you reach your capped limit, surfers are turned away or are greeted with a slow, almost impossible site to use. No one can make purchases on your site, or paying members cannot access your wonderful content, with the end result being LOSS of profit and revenue.

Example #2: You have a site that offers articles or content that others web sites are putting onto their web sites (and possibly paying you for your content) and your site gets listed on digg.com or is discussed on the major news outlets, thus getting hammered with traffic and bandwidth goes up. If you are capped, then it bottlenecks the site and surfers have to WAIT for the page to load until the people in line ahead of them are done...then your clients start complaining, possibly start canceling their service, because their surfers are complaining. Obviously, no one likes losing paying customers.

Of course, the previous two examples make the assumption that the traffic hitting your site is good quality traffic. However, there are times when your bandwidth is being stolen via hotlinking (another web site linking directly to images/content on YOUR server) and this is not desirable. At NationalNet, our monitoring system will alert us to abnormally high bandwidth, our system administrators will investigate and stop the thieves as well as notify you of this high bandwidth. However, you should make it a habit to check your website stats on a daily basis. Not only will this help you understand your traffic and visitors better; you'll catch any large bandwidth jumps before they can be too costly for you.

So, as you can see, having a capped (unmetered) connection is probably not what you want. The key is making sure that you check your bandwidth stats on a regular basis and that you utilize a host that will watch it for you as well and alert you if the bandwidth starts to exceed your budgeted amount. Also, make sure that you know what bandwidth overages will cost you. Many hosting companies advertise their plans on their site and will list a server and some amount of bandwidth for X amount of dollars, but no where in the plan details does it list what the overages are, and in many cases, those overage charges are considerably more than what the regular commited rate is. Be sure you know what those charges are.

Now, with all of that said, there are times when an unmetered plan is exactly what you want. If you have a site that you know will never exceed your plan unless something really bad is happening, or you have a site that is not revenue driven, or you don't really care if it's slow at times, than an unmetered plan may be exactly what you need. Unmetered plans tend to be cheaper as well, due to the fact that the hosting company knows exactly how much bandwidth they must purchase and do not have to purchase extra bandwidth to cover overages and spikes.

Traffiic/bandwidth by its very nature is very spiky. On any given day, it goes up and down in fairly wild extremes. For instance, our own bandwidth graphs look like mountains and valleys. Joe surfer gets out of work, and the bandwidth goes up...and keeps going up until about midnight EST, when it starts going down. Special traffic deals, viral marketing, etc, all contribute to this "spikiness" (did I just make up a word?) Any host worth it's salt must make sure that they have lots of extra bandwidth overhead to cover this spikiness, so that the actions of one or two webmasters does not affect everyone else.

It's very expensive for a good host to pay for all that "bandwidth overhead", but in the long run, it's well worth it.

One final thing to be aware of regarding unmetered/capped plans is that many times these plans are on shared bandwidth. What this means is that the host or provider is actually capped themselves by their upstream providers, or that they have purchased a set amount of bandwidth and continually add customers to this set amount and hope that their customers never use all of the allocation. This is commonly called "overselling". A good example is a host that has a 1 Gbps connection to their provider but sells 200 10 Mbps plans (the equivalent of 2 Gbps) on that single connection. The risk here is if even ½ of their customers use their entire allocation, all customers are going to suffer due to the lack of bandwidth to go around. Overselling is a risk that some hosts take, but NationalNet will never take. It's not worth risking our reputation by having even one day where the network is slow due to overselling.

What is this 95th Percentile (or, the difference between throughput and transfer)?

Many, if not most hosting companies sell and bill bandwidth based on a method called the 95th percentile. Many, if not most customers, don't have a clue what the 95th percentile really is. In this article, I'll try to shed some light on what 95th percentile is.

In order to explain, we must first understand the difference between the two types of bandwidth billing methods. Those two types are TRANSFER (95th percentile billing) and THROUGHPUT (per-gig billing). Let's look at them individually....

Throughput is the actual total SIZE of the combined files that are sent by the server. Throughput is sold in Gigabytes (GB) and is an aggregate monthly total. So, for example, let's say you have a web page called THISPAGE.HTML and the actual page is 25k, on this page you have 3 graphic images that are 25k each which is a total of 100k. If 100,000 people downloaded that page over the course of a month then your Throughput would be calculated as 100kB X 100,000 = 10,000,000kB or 10GB. So for that month your THROUGHPUT would be 10GB. This does not take into account if all 100,000 people hit the server at the same time or were evenly spread out over the course of the month; it is still 10GB of THROUGHPUT for the month.

Now, on to TRANSFER, but before we begin let me state that in *NO* circumstances can you mix Throughput and Transfer. It is physically impossible (it's like trying to add up gallons and nickles). They are two different things.

TRANSFER is measured in Megabits Per Second (Mbps) and measures how much information is traveling through the Internet "pipe" at any given time. I like to compare TRANSFER to water in a series of water pipes. Imagine that your home PC has a water hose connected to it instead of an Internet connection. The water hose is 1/2" and is connected to the side of your house where it meets a 2" pipe and your house is connected to the Water Main, which is a 12" Pipe. In this example your ½" water hose is your home Internet connection and your 2" pipe to your house is your ISP and the 12" water main is the backbone of the Internet. It does not matter how hard you try you are only going to get 1/2" of water into your PC at any given time because the "pipe" is only a 1/2" water hose.

Now if I were going to sell you water BY THE GALLON, that would be called Throughput (see above), or I can sell you a PIPE and just charge you for the amount of water that you push through the pipe at any given time...this is called TRANSFER. For example, if I take a measurement right now and you are pushing 1" of water through the pipe and I look again in five minutes and you are pushing 1" still and I look again in five more minutes and you are pushing 1/2" and I look again in five more minutes and you are pushing 2" then how big of a pipe do you need to accommodate your traffic flow without any water being backed up like a funnel??? You would need a 2" pipe, but you are not using 2" all the time, so why do you have to pay for a 2" pipe all the time?? This is where the 95% comes in.

The 95th percentile (which is an industry standard) simply means that the hosting company will look at your pipe every five minutes and take a reading and add that reading to a long list that they keep for 30 days. At the end of the month that list will contain 8640 readings (there are 12 five minute intervals in an hour, 24 hours a day for 30 days). They will then take that list and sort it from the biggest number to the smallest number so that your largest five minute reading is on the top, the second largest is next, the third largest is next and so on. The top 432 entries (the top 5%) are discarded and the 433rd is considered your "95th Precentile" and that is the number that you pay for. The 95th percentile was designed to help chop off wild peaks and only bill you for what you are sustaining on a regular basis. This is a rolling 30 day number that is constantly changing. In other words, once you get the 8640 data points, every time a new data point is added and the list is sorted, the oldest data point is dropped off.

As for what is more advantageous, it depends on the traffic patterns of your site. THROUGHPUT (95%) is good for almost all sites with very few exceptions. TRANSFER is recommended for sites that have extremely high spikes or very inconsistent traffic. For example, if you have very high traffic every Monday but the rest of the week is very low traffic, then being billed on THROUGHPUT may be the best for you. In this case, you would have lots of big numbers due to that high traffic on Monday, which would create an inflated 95th percentile. However, very few sites have this type of traffic pattern.

With TRANSFER host should provide 95th percentile graphs (usually MRTG graphs which is the industry standard) and you can see your transfer yourself. You should check these graphs every day as they can indicate problems as well as let you know your traffic patterns. You should see highs and lows each day and these patterns of highs and lows should follow the sun. If you see a flat line across the top of the graphs then you know that your hosting company doesn't have enough bandwidth to handle your needs (and this is much more common than one would think). ***IF YOU ARE BEING BILLED ON 95TH PERCENTILE MAKE SURE YOUR HOSTING COMPANY PROVIDES YOU WITH THOSE GRAPHS*** If they refuse, they obviously have something to hide.

Hopefully this helps you understand what 95th percentile is.