Windows 10 – 2004 – Benchmark - Optimized with VMware’s Operating System Optimization Tool (OSOT)
If history has told us anything, it’s that the performance & scalability of Windows 10 releases can fluctuate a lot.
Combine that with the fact that the releases come in fast (as a subscription), and potentially there’s a recipe for disaster.
The good news: There are free tools available that can improve Windows 10 performance significantly, but the question remains: how effective are they, and do they break anything else in the process of removing software and services? Ever since the first releases of Windows 10 at Login VSI, we’ve published a report on scalability, and this release is no different. However, our software can also report back on compatibility, so if you want to know if all your applications still work after an upgrade, request a free trial.
For these tests, I’ve set up a test lab (Cisco Hyperflex) that’s easily capable of running 250 users in a standard virtual machine configuration with 2CPU’s and 4GB of memory. I’ve then used Login VSI’s software to simulate 250 users within the timeframe of approximately an hour using the industry-standard knowledge worker workload.
We’ll focus on three things in this post:
- The maximum number of users on a machine or environment before the performance starts to degrade
- The performance of the machine or environment with little to no stress
- Logon times
- Logon times for users accessing the environment
Clean – No Optimizations
Let’s set a baseline, how well does Windows 10 perform without optimizations? The answer is: Not great; Windows 10 has been designed as a consumer operating system with all the bells and whistles that anyone could wish. However, if all these extra’s account for just 2% CPU utilization per user, you’d be in trouble if you had 50 people working together on the same machine.
In the clean scenario, the only change made to the base image is the installation of the OneDrive machine-based client rather than the User-based client, as this gives a very significant boost in performance. I assume that Windows Virtual Desktop drives Microsoft to work on increasing the performance of these solutions as they are now hosting their solution, ultimately benefitting even the on-premises customers.
Image 1: Login VSImax (Higher is better)
It's clear to see that, over time, the new features added to the operating system, causing the scalability to fluctuate quite a lot. In theory, all 03 releases (Spring) should be about new features where 09 (Fall) releases are focused on security and performance. On average, we can say that Windows 10, clean, without optimizations, can run 150 users on my lab-environment.
Image 2: Login VSIbase (Lower is better)
The second parameter to look at is the VSIbaseline; storage systems, anti-virus, and workspace managers typically force the numbers to be as low as possible. We can see that with the most recent releases, Microsoft is trending in the right direction again.
Image 3: User Logon times (Lower is better)
Lastly, for the non-optimized tests, let's look at the Logon times, there are apparent differences in Logon times as we are getting closer to the 250-user mark on the y² axis. Windows 10 1909 seems to be the odd one out, whereas, with Windows 10 2004, things seem to be back on par with Windows 10 1903 again.
VMware’s OSOT has been around since June 2014, and the team has gone through extreme measures to find, create, test, and optimize every bit of Windows 10 they could find. I’d argue that they’ve run more Login VSI tests than I have...and that’s a lot! At the time of testing, version b1151 was publicly available. Recently they’ve released version 1160 that from initial tests seems to perform even better than outlined in this post. I’ll make sure to get you an update on that. As of April 2020, the VMware optimizer officially supports Windows 10 2004.
Let’s look at the numbers:
Image 4: Login VSImax non-optimized vs. optimized Windows 10 (Higher is better)
It's clear that the VMware Optimizer has a very positive impact on performance; the average number of sessions on my environment increases from 143 to 246, making the time it takes to run the VMware Optimizer (5 to 10 minutes) worth it. Keep in mind that any optimizer is tinkered with default services and apps - that’s why I highly recommend checking if all your applications still work with Login Enterprise’s Application Compatibility mode. Even if you have thousands of applications, it’ll quickly let you know if they’ll all still work after your changes.
Image 5: Login VSIbase non-optimized vs. optimized Windows 10 (Lower is better)
There’s also a positive impact on the baseline, where it's about 18% lower than before running the optimizations. Users will notice the positive effects of this even when your environment is not saturated.
Image 6: User Logon times (Lower is better)
Analyzing the logon times, it's clear that there is a significant improvement overall; if we compare the logon times of Windows 10 2004, without optimizations (the red dots) to all versions of Windows 10 with VMware’s optimizations its clear there is a very significant gain after optimizations. I’d say this is a textbook example of how performance should be.
It's fair to say that there are vast benefits to using the VMware Optimizer. Whether you want to do that on your VDI environment, your more traditional Server-based computing hosts, or even your PC, you’ll probably notice an increase in performance. There is a newer version of OSOT, so keep an eye on it for updates.
Image 7: Click to enlarge
Special thanks to Neda, Lorna, and Blair for their help performing these tests and spreading the word at various leading industry events. If you like, here’s a recording of the webinar we did together on these results.