Suppose you have important business information that you want to protect with a secure backup to the Internet. One of the many questions you’re likely to ask a potential vendor is:
How long will it take to actually perform the backup?
In this blog post we’ll see what’s actually involved in answering this common question — which at first blush sounds simple. But, as you will see, there are quite a few variables and technical assumptions we must make to even come up with a ballpark answer.
So let’s gather some sample data, make our assumptions and come up with an estimate. Then, we’ll see how long the backup actually takes in the real world.
Making our Initial Estimate
Let’s say the data we want to safeguard is located inside of the My Documents folder. By right clicking on that folder name in Windows Explorer and selecting the “Properties” menu option, we observe that (at this moment) there are approximately 3,800 files in 458 folders using about 9 GB of disk storage.
If we assume continuous Internet upload speeds of 1 Mbps, the speed calculations say that we can transfer approximately 400 MB (0.4 GB) in an hour.
Note: Approximately 100 MB can be transferred per every 256 kbps of available up-link capacity.
Crunching the raw numbers gives us: 9 GB / 0.4 GB per hour = 22.5 hours ~ 1 full day.
Given this calculation, our initial estimate is that it would take almost a full day to transfer the files to the Internet over a reasonably fast Internet connection. So, let’s set things up and see how this actually plays out in the real world!
The Live Test
Early the next day, we fired up the Dr.Backup client software, configured and started the backup job we’ll use to test our estimating methodology.
As shown in the screenshot below, the backup set called “Calculate My Document Backup Time” completed its initial file scan and began transferring data to the Internet.
We see that today there are a few less files in the folder from when we originally looked – but about the same amount of storage used – so our testing stats are still valid.
The curious “geek” inside me wanted to actually see just how fast data would transfer up to the Internet while the backup was in progress. To get this information required the use of a live bandwidth monitor – which just happen to already be installed on my computer!
A snapshot from the real time bandwidth graph (below) lets us observe the instantaneous up-link transfer speed topping out at just above 2 Mbps. This is consistent with the predicted burst rate of the Dr.Backup Professional Package service.
Given the increased upload speeds that appear possible, it follows that this initial backup could actually take less time if the increased speed could be sustained over an extended period.
Throughout the day, things progressed pretty much as expected. The backup ultimately completed and the screen looked like this:
Performance Analysis
Once all of document files had been backed up, a convenient summary email arrived from Dr.Backup.
Let’s take a look at the summary statistics on the email and see how our backup job actually performed. We can then compare that to our estimate and see how we did.
==================================================================== *** Dr.Backup Online Backup Service - Backup Notification *** Prepared: Monday, May 12, 2014 5:33:57 PM (GMT-4) ====================================================================
--------------------------------------------------------- Username: drbackup-10 BackupSet: Calculate My Documents Backup Time Destination: Remote Server --------------------------------------------------------- Status: Backup Finished, but with Exceptions (Exception Count: 1) Please review your backup log. Warnings detected. --------------------------------------------------------- Files Transferred: 3,766 Bytes on disk: 9,598,324,285 Compressed to: 7,016,661,576 (73% of original size) Completed: 5/12/2014 5:33:24 PM (local time) ---------------------------------------------------------
The email text shows that we did indeed backup just under 3,800 files representing over 9+GB of data – and that jives with Windows Explorer and the progress graphs shown in the Dr.Backup client.
Note: The backup log (not shown) revealed that one small file was actually deleted before it could be backed up – which explains the exception count.
The backup started at 9:45 am and completed at 5:30 pm local time. So the total run time clocked in at about 7 hours and 45 minutes.
Hmmm. That’s a lot less than the 22.5 hours we had estimated — so what gives?
Looking closer, we can see that data compression was employed in the backup which resulted in our original data being compressed down to 73% of its original size BEFORE it was actually transmitted over the Internet.
This pre-processing, while adding “overhead,” actually resulted in a meaningful performance improvement in the backup timeline.
So technically, we were able to transfer 7 GB of compressed data in 7.75 hours, about 900 MB per hour. This equates to an upload speed of a little more than 2 Mbps – also, significantly higher than the 1 Mbps we originally estimated.
So, as it turns out, our time estimate was a bit on the high side – but we now have a pretty good understanding of why.
Summarizing What We Learned
Repeating this test several times allowed us to pretty much confirm the results we observed above. We were also able to determine that some of the key factors important to the performance of the online backup were:
Data compression – Faster PCs can pre-process data quicker making backups run a little faster
File size – A smaller number of large files backs up faster than a larger number of small files
Backup Schedule – More businesses backup in middle of the night, so it’s a little slower
Internet connection – Up to a point, faster connections perform better
All things considered, we were able to protect 9.5 GB (3,800 files) of data in just under 8 hours, yielding an effective throughput rate of just over 1 GB per hour. Average data compression on the test set was about 25%.
If we assume that a typical business has a backup “window” of approximately 10 hours, for best results, nightly online backup volumes on this computer should average less than 10 GB.
Please keep in mind that in most cases, only new data files (or files that have changed since the previous backup) need to be processed each day.
Finally, while the process we just went through was useful, it’s not definitive. Every situation is different.
The performance of your online backups could be significantly different than what we observed here – but at least you now understand the methodology used and can perform your own sample testing.