GUIDE

VPN Speed Test Overview

To start this VPN speed test article, let’s note that this article deals solely with measuring OpenVPN performance, as to be honest this is the only VPN protocol we really care about, as the others are rubbish (well SSTP is ok, but it is limited to Windows computers only, is poorly supported by VPN providers, and offers nothing that OpenVPN can’t do anyway). Please see here for more details.

When doing VPN tests to measure connection speeds, most people turn to the web’s most prominent internet speed testing website, speedtest.net. Unfortunately, although speedtest.net is a very good indicator of naked internet speed, it can yield some very odd results when VPN testing, often indicating speeds much faster than not just the naked internet results, but than the cap put on a service by the ISP .

The reason for this is that the Flash based speedtest.net tool does not account for LZO compression, which is built into the  OpenVPN protocol. We will discuss this in more detail below, but what it does show is that we need a more accurate way to measure data transfer speeds over OpenVPN.

Factors that influence connection speeds

ISP speed is only between you and ISP server

When ISPs offer a set broadband speed, this only applies to the connection between your computer and the ISP’s nearest server, after which data transfer between ISPs and countries is a free-for-all that no ISP has any control over.

This is not a fact that ISPs like to boast about, but is a necessary consequence of how the internet works. More than anything else, this factor can lead to highly inconsistent results, but there is little that can be done about it other than performing many VPN tests, and averaging the findings.

LZO compression

OpenVPN has built into it the optional ability to use the LZO lossless compression library. Much like the better known .zip format, this compresses the size of some files types, which can indeed increase data throughput, but does not count against your ‘real’ bandwidth usage. Files that are already compressed, such as .zip, .rar, and .mp3, and .jpg files, do not benefit much from additional compression. As we noted earlier, speedtest.net is easily confused by the use of LZO compression.

TCP and UDP

OpenVPN can be run over either the TCP (Transmission Control Protocol) or UDP (User Datagram Protocol) transports. TCP uses error correction to verify that a data packet has arrived before resending the packet (if no confirmation is received), or sending the next packet (if confirmation is received). UDP on the other hand doesn’t care, and keeps on sending packets regardless of whether they have arrived or not.

UDP is consequently faster (but less reliable) than TCP. See here for more details.

ISP throttling

Also known as traffic shaping or bandwidth limiting, most ISP do this to some extent (and in the US the whole notion of net neutrality, where all internet traffic is treated equally, has been effectively trashed – at least for now).

BitTorrent traffic is regularly throttled, but subtler shaping due to peak usage times, and other factors is common. If you want see how much throttling your ISP is responsible for, check out M-Labs transparency measurement tool.

VPN tests

Our UK ISP is Virgin…

Programs that hog bandwidth

When trying to perform an accurate VPN speed test, it is important to check that you are not running background programs and processes that are hogging your bandwidth and slowing down your connection.

This is particularly a factor with uploads, as while most ISPs are fairly generous with download limits, upload bandwidth is typically much more limited. As a rule of the thumb, using more than 1/4 of your upload speed will noticeably slow down your download speeds.

BitTorrent clients are particularly notable culprits for hogging upload bandwidth, but you should also keep an eye on online storage services such as Dropbox and Google Drive (and pause them during bandwidth speed testing).

VPN overhead

VPN always comes with a performance hit thanks to the fact that data often has to travel further between your computer and the internet, as it has to go via a VPN server. This problem is made worse the greater the distance you are from the VPN server, so you should always do your VPN testing using the server geographically closest to you.

In addition to this, the encryption and decryption process takes up processing power on your computer, which may in practical terms affect speed (although it should not affect these tests). This is not a major problem on most modern desktop/laptop computers, but may affect low-end smart phones and the like. It is also worth remembering that the more encryption used, the more processing power is required.

More importantly, handling VPN traffic tasks a VPN server, and insufficient processing power at the VPN server end of things can significantly slow down connection speeds.

In theory, when connected to a nearby VPN server using OpenVPN, we should typically expect to see around a 10 percent loss of speed compared to not using a VPN (this at least is a commonly quoted figure, and is bourn out by our testing results (see this article). These results however, have mainly been obtained using  speedtest.net, so as this article discusses, may not be as accurate as we would like)

VPN speed testing methods

Tests were performed using a 20Mb/s UK broadband connection, and for VPN using TorGuard. Our methodology was to run three tests  for each test method without VPN, and then three with VPN , in quick succession so that factors beyond our control (the speed of the internet) were as consistent as possible.

Ideally, we would like to have tested the different methods as simultaneously as possible, but real-world considerations mean that our results were obtained over a couple of days. The idea was to do some measurements, crunch some figures, and see what conclusions we could draw from the results.

Speedtest.net

All speedtestnet results

Here the first (lower) three results are without VPN, followed by results when connected to TorGuard. On average download speeds were:

  • no VPN = 20.1Mb/s
  • with VPN = 22.9Mb/s,
  • VPN = 12.1% faster.

On average upload speeds were:

  • no VPN = 1.3Mb/s
  • with VPN =1.1Mb/s
  • no VPN = 15.4% faster.

Here we can see clearly see the problem with speedtest.net, as the download results are notably higher than both the no VPN results, and our ISP imposed speed cap, which is simply not possible were the results accurate. LZO compression is almost certainly to blame for this.

Testmy.net

Testmy.net uses the more robust HTML5 for its speed tests, and claims to compensate for compression. Unfortunately, the closest test server to our UK location is in the Netherlands, so some slowing down due to lag is to be expected.

testmynet no vpn results

No VPN

vpn speed test

With UK VPN

vpn testing

As the Testmy.net test server was in the Netherlands, we also thought we would see how using an NL VPN server affected the results.

Average download speeds were:

  • No VPN = 16.4Mb/s
  • with UK VPN – 16.4Mb/s
  • with NL VPN = 13.7Mb/s
  • UK VPN = same as no VPN, while NL VPN = 16.5% slower than no VPN

Average upload speeds were:

  • No VPN = 1.1Mb/s
  • with UK VPN = 1.09Mb/s
  • with NL VPN = 1.06Mb/s
  • UK VPN = 1% slower than no VPN, while NL VPN = 4% slower than no VPN

It has to be said the results are rather inconsistent, although the fact that the test server was so far away from us may well have contributed towards both these issues. On average, the NL VPN result was actually 0.01Mb/s faster than not using a VPN, but this is a minor discrepancy, easily explained by the chaotically variable nature of the internet, and by the inconsistency of our results.

What is clear in this VPN speed test is that we achieved better speeds when connected to our closest VPN server, rather than one close to the test server.

Speedof.me

Another HTML5 web-based speed test, Speedof.me automatically selects the test server based on stability, rather than distance. It chose London for these tests, which is the closest anyway, and should give us the most accurate idea of real internet speeds.

speedofme no vpn

No VPN

speedofme wth vpn

With VPN

Average download speeds were:

  • No VPN = 18Mb/s
  • With VPN = 8Mbs/s
  • VPN = 65.6% slower than no VPN

Average upload speeds were:

  • No VPN = 1.2Mb/s
  • With VPN = 1.9Mb/s
  • VPN = 37% faster than no VPN

Other than TorGuard having a terrible day for download speeds, the overall results show some of the best consistency we have seen so far. An exception is the highly anomalous 3.25Mb/s upload result, which approaches three times the speed of every other upload test so far, and gives us pause when considering the overall accuracy of the speedof.me tests.

ttbMeter

Unlike the previous tests, ttbMeter from the UK’s thinkbroadband service (‘an independent broadband news and information site helping you to get the most out of your broadband) is a downloadable tool with a ton of configuration options and tests available. Unfortunately, many of the tools did not seem to work, so we opted for a straight download test using a 100Mb file.

tbbMeter no vpn a

No VPN

tbbMeter with vpn 2

With VPN

Average download speeds were:

  • No VPN = 16.2
  • With VPN = 17.4
  • VPN = 7% faster than no VPN

As a straight timing test for a 100MB file, these results are highly consistent. Again, the very slight discrepancy in speeds being faster with VPN than without can be explained by variable factors. On a practical level, however, we found tbbMeter frustrating to use, as it often appeared not to work, and only displayed the results briefly as a notification bar bubble.

Manual VPN speed test

When it comes to testing internet speeds there is nothing wrong with going back to basics. This series of tests simply involved downloading a test file, and using a stopwatch to time how long the download took.

When performing such tests it is important to use compressed files to compensate for LZO compression. We u

Without VPN using a 100MB test zip file, courtesy of thinkbroadband (100MB = 800Mb)

  • Result 1: 800Mb / 50.92 seconds = 15.7Mb/s
  • Result 2: 800Mb / 71.32 seconds = 11.2Mb/s
  • Result 3: 800Mb / 76.35 seconds = 10.5Mb/s
  • Average = 12.5Mb/s

With VPN

  • Result 1: 800Mb / 80.04 seconds = 10Mb/s
  • Result 2: 800Mb / 78.33 seconds = 10.2Mb/s
  • Result 3: 800Mb / 91.16 seconds = 8.8Mb/s
  • Average = 9.7Mb/s

So no VPN is 22.4% faster than with VPN.

We also uploaded a 5MB (40Mb) file to Dropbox:

  • Upload without VPN: 1.08Mb/s (36.9 seconds), 1.09Mb/s (36.55 seconds), 1.08Mbs (37.09 seconds). Average = 1.08Mb/s
  • Upload with VPN: 0.96Mb/s (41.85 seconds), 1.09Mb/s (36.55 seconds), 0.77Mb/s (51.66 seconds. Average = 0.95

So no VPN is 12.2% faster than with VPN.

Although it lacks a certain precision (hitting the stopwatch and download start buttons at exactly the same time is tricky, and uploading to Dropbox is not the most clinical method), this test does give a very ‘real world’ measurement of internet connection speeds, and how VPN affects them. Interestingly, it generally gives us much slower download times than the automated tests.

And finally…

So taking all these figures together (except the speedtest.net one which is definitely out) , we find that:

  • Average download speed without VPN = 15.8Mb/s
  • Average download speed with VPN = 11.3Mb/s
  • Average upload speed without VPN = 1.1Mb/s
  • Average upload speed with VPN = 1.3Mb/s
  • On average downloads are 28.5% slower using VPN
  • On average uploads are 7% faster using VPN (clearly an anomalous result)

Tesing VPN Speeds Conclusion

The biggest take-away from these tests is how inconsistent the results are, not just between different test methods, but when performing identical tests back-to-back with each other. It therefore has to be said that any published speed test results (including those on the BestVPN website) should be taken with a big pinch of salt.

This is not to say they cannot be useful for providing a general guide to VPN performance, but only after performing a large number of tests over a period of time can any very accurate picture be gathered. Other things we observed during the tests include:

  • TorGuard’s results were rather lackluster (other providers should hopefully yield better results)
  • For best speed results you should connect to a server near you (even when connecting to a test server further away)
  • The manual approach probably gives the best results, but is too impractical for daily use

Because of its relative consistency and ease of use, BestVPN will use Testmy.net from now on, but please be aware of the inherent limitations of such tests.


Douglas Crawford I am a freelance writer, technology enthusiast, and lover of life who enjoys spinning words and sharing knowledge for a living. Find me on Google+

Related Coverage


12 responses to “VPN Speed Test Overview

  1. I add my voice into this discussion. I use 65/4,5 Mbps cable internet link. When I test it with speedtest.net I have every time this very same measures (up to less than 5%).
    I also use VPN AT&T corporate connection. Speettest results are amaizing: 2Mbps download and 20 Mbps upload.

    I discussed this issue with some guy working for this corporation. He explaining was:
    1) Download is strictly limited by company policy. 2Mbps is considered to be acceptable for all remote users.
    2) As the skeleton in my country is owned by AT&T, the provider is obliged to direct the AT&T traffic through the unrestricted channel.
    All remaining jobs are conducted by corporate proxy.

    Hard to belive.

    Tomek

  2. Problem with testing it on a 20mbps is that, thats a relatively slow speed for today’s standards so many VPN servers can match it but doesn’t mean they’re “fast”….for example I’m on a 80mbps bandwidth and none of the VPN services that i have tried go beyond 40mbps so they would come up as “fast” in your tests because they have no problem matching your 20mbps bandwidth but they’re all very slow for me because they can only match 50% of my bandwidth.

    1. Hi Mike,

      Well, at the time of writing this, a 20mbps connection is all I had. I now have a 50mbps connection, and FWIW, regularly (but not always) get 46-47mbps using AirVPN (from UK connected to NL server…)

  3. I am in Kerala, India. My data files are in Abu Dhabi, UAE. I am not able to work with my files even the local net speed is 20 Mbps. My VPN signal strength is poor. Kindly suggest me what to do to get my VPN speedy so as to work with my files from UAE

    1. Hi Chandra,

      Are you using a commercial VPN service (designed to give you privacy and security on the internet), or are you using VPN to connect directly with a server in the UAE? If using a commercial service, are you getting quick access to the UAE without the VPN? If so then you should contact your VPN provider about the issue (or change provider).

    1. Hi Jeff,

      You may want to check out Why we’re changing our VPN Review speed tests. The problem with performing more frequent tests is that a) doing so is very time consuming, and b) requires re-signing up for a given service each time. The BestVPN website is currently going through a period of rapid expansion, however, and as our resources grow we may consider looking into this issue in greater detail…

    1. Hi Tony,

      Thanks for those great links! Yeah – basically you can only compress the ones and zeros that make up data so much…

  4. Very interesting, thanks for the tests and write-up. I’m on a VPN server 24/7 that is over 6000 miles away and wonder to what degree LZO helps, along with the true cost of the VPN overhead. Not that I would ever find time, but it would be interesting to run enough tests in a lab to get the true cost of running VPN with, and without LZO enabled.

    You mentioned that already compressed files such as .jpg or .mp3 don’t benefit from LZO. From my understanding of LZO, the data type is completely irrelevant as the VPN process is fed a specified amount of data based on the MTU size. When it has the data to be sent, it performs compression for that packet and it is irrelevant what the data is, or the amount of compression that has already occurred since it is in a new context. On the other end, it is decompressed by the VPN client.

    1. Hi Tony,

      My understanding is that data can only be compressed so much (at least without losing information, and LZO is a lossless data compression algorithm). I do like your suggestion of about LZO comparison tests though, so this is a subject I may revisit in detail the future. Thanks.

Leave a Reply

Your email address will not be published. Required fields are marked *