Beta functionality

From Login VSI Documentation
Revision as of 14:58, 23 August 2016 by Admin (talk | contribs) (Login VSI 4.1 Beta Program)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

This page will describe the new beta functionality.

Login VSI 4.1 Timer Metrics (BETA Workloads)

These metrics are used in our upcoming Login VSI 4.1 release but are used in our BETA workloads that are available from Login VSI version 4.0.10 and up.

Measurement ID

Measurement Action

Measurement Action Detailed

Measures Related Resource

NSLD

Notepad starts and loads a 1500kb document

Notepad starts and loads a local 1500kb document file that is randomly copied from the content pool

CPU and IO

NFO

Measure how long it takes to show the file-open dialog in VSI notepad

VSI-Notepad file open [ctrl+o]

CPU RAM, and IO

ZHC*

Create a zipfile with high compression

Compress a local random .pst file that is copied from the content pool (5mb)

CPU and IO

ZLC*

Create a zipfile with low compression

Compress a local random .pst file that is copied from the content pool (5mb)

IO

CPU

Calculates a large array of random data

Creates a large array of random data that will be used in the IO timer

CPU

'*'compression is measured with 7zip

New 4.1 (Beta) Analyzer Tabs

These new tabs contain information based on the new VSImax 4.1 (Beta) calculation metrics.

Description

Screenshot

These tabs show information for the individual measurements taken during the 4.1 beta tests. These specific measurements are zoomed in to because they are used to calculate VSIbase and VSImax v4.1 threshold.


The tab is similar to the VSImax v4 detailed tab except that it, by default, will only show the measurement for the tab. The graphs scale has also been scaled for the individual measurement.

The tab will also display the baseline value for this measurement. This is the time it takes to complete this measurement during baseline measurements. These measurements are taken while the system is under no or very little load. It is used to see how the measurement trends from a system that isn’t under load.

NSLD 4011 Analyzer Tab.jpg
NFO 4011 Analyzer Tab.jpg
ZHC 4011 Analyzer Tab.jpg
CPU 4011 Analyzer Tab.jpg
IO 4011 Analyzer Tab.jpg
UMEM 4011 Analyzer Tab.jpg

These tabs also allow you to add any of the other metrics. The baseline for the tabs specific metric will not disappear though.


Login VSI 4.1 Beta Program

Login VSI 4.1 Beta Preview now available for download

Checkout the new Login VSI 4.1 beta features which makes it easier to locate VDI performance bottlenecks and features even more realistic workloads. In this beta release the following new features and workloads are available for preview:


News and improvements Two workloads are added; the “Knowledge worker” for 2vCPU desktops and by popular demand the “Office worker” for 1vCPU desktops will be reintroduced. Both workloads are based on the Medium workload introduced in Login VSI 4.0.

The main difference between the “Office” and the “Knowledge worker” is the reduced CPU usage by removing the video’s and the freemind (java) application. The office worker profile is focused on broad Office and Internet Explorer usage and is designed to run within 1vCPU virtual desktops.

Login VSI will recommend the knowledge worker as the standard and is designed to run in 2vCPU desktop to properly support the web video content.

In 4.1 the relative weight between the different workloads is more evenly balanced to better reflect real-life usage. The video library has also been improved, all video formats are now generated from the same source files. The footprint of the Video library is now considerably smaller, which makes the installation and setup of the VSI data/web server easier.


Locate performance bottlenecks Login VSI 4.1 introduces the ability to import data generated by Perfmon, ESXtop, or any other source csv in the Login VSI analyzer with ease. This allows you to correlate performance data with VSImax to easily identify and understand the bottleneck of the entire environment. The time sync feature helps to import log files with timestamps that are not in sync, this is a great timesaver as log files of different systems no longer have to be merged manually.

The goal of this beta is to collect as much data on the workloads and the timers as possible so that we can fine-tune the timers and workloads before they are marked as final.

Login VSI 4.1 is now available for download at https://www.loginvsi.com/downloads/login-vsi.

Login VSI 4.1 Beta Analyzer

How to import external data with Login VSI 4.x PRO


This beta feature combines the functionality of an external application, for example ParseESXTop, that is used to make graphs of csv's created by ESXTop, and the VSIMax graph. All the data inside the csv of ESXTop, Perfmon, or any other monitoring tool that is used, is either in percentages or recalculated to percentages.

Login VSI 4.1 Beta Analyzer

  • Import (multiple) external performance data in analyzer
    • Perfmon,
    • ESXtop,
    • Xentop
    • All other csv file based logging
    • Easily overlay graphs and fix time-offsets
  • Show VSImax results weighted (relative to each other)
  • Ability to merge test results to calculate and save average over multiple tests
  • Ability to compare tests to eachother
  • Ability to see used memory (per vm)
  • Ability to visualize application start times
  • Workflow improvements
  • UX improvements
  • Added: Extra tab in the analyzer with information about the test


Analyzer4.1.PNG
The blue steps line in the picture is representing the active sessions. Everything is recalculated to time in seconds, so the X-axis is time in seconds. The left Y-axis are the values of the VSImax information graph. Normally approximately 0-10000 and is in milliseconds.


  • Auto scale


All the external data that is imported, must fit between the scale of 0 and 100 (the right Y-Axis in the External data chart). To get this work, any metric needs to be scaled down (or up in some cases). It finds the highest metric and determines which scale factor it has to use. When the highest metric is between 100 and 1000, it is divided by 10, when the highest metric is between 1000 and 10000, it is divided by 100 and so on. If a scale factor is filled in in the XML template, that one is used as well. By default this is set to 1.0.


  • Graph sliders


To correct time differences, or just to play around with the series in the external data graph, there is a custom user control created. For every source of external data in the graph, a Graph Slider is created in the Beta functionality. To link the sliders, click the "Link all sliders" checkbox and all the checkboxes with the sliders are checked. When two or more sliders are connected, they all behave the same as the operated slider. That is, when the checkbox is checked.

Analyzer4.1.autoscale.PNG


  • Combine VSImax Graphs
Combined tests.PNG


At this point we have a functionality to average out two or more graphs into one. It averages out all values from calculated table and creates a new average table from where the new graph gets all values.


Combine (create avg) 4.1.PNG


If you have combined data it is possible to save these generated graphs. The save dialog can be found under save as in file menu if the combined test is active.


Combine (create avg) save as 4.1.PNG


Function save, saves the tests in the following combination (full data, raw data and baseline raw data). Also it adds a new table 'SavedLogs' with the ID number of the test that were combined. The function is extended to include the configuration files of all the test that are in the test.


  • Compare logs

When starting the Analyzer or open the load dialog to open log files, you get an option to compare different log files. Comparing logs now includes all the VSImax graphs of every log entry in one chart. To make the chart less messy to start with, only the VSI average index series are shown for each log entry by default.


Compare chart.PNG
Compare-vsimax overlay.PNG


Weighted VSI metrics

The analyzer has had a tab that shows the individual metrics measured by Login VSI. Since the scale of these metrics go from very small to very high which makes it hard to compare them. The VSImax V4 Detailed Weighted tab will show the same metrics as the VSImax V4 Detailed tab with their weights applied. This will make it easier to compare the different metric.

The default weights can be found here: Start your first test

Login VSI 4.1 Beta Analyzer Importing external data

Welcome to the Login VSI beta program. We are happy to share with you our early beta release in which you can import External Performance data like ESXtop or Perfmon in the Login VSI analyzer. The Login VSI 4.1 Beta Analyzer is shipped with the Login VSI 4.0.8 installation package.

All metrics are down/up scaled to be in the range of 0-100, by finding the max value and multiplying accordingly. This feature was expanded to be able to hold multiple sources and multiple imports of metrics with ability to create new tabs or merge to the existing external data tabs. This also adds the remaining external data series to moving box, and adds the source column in the data grid view.


Please do not use this analyzer to publish any results, use only for testing purpose.


Description

Screenshot

Open the Analyzer located in {VSIshare}\_VSI_Binaries\Analyzer\Login VSI Analyzer.exe or run the Analyzer from the Management Console.

Manual open analyzer.PNG
Auto open analyzer.PNG

Specify the {VSIshare} and click save.

2013-11-04 01 52 21-Login VSI Analyzer.png

Open a result.

Open result.PNG
Analyzer before import.PNG

Click file – import – external data or shortcut key [ctrl+x]

Analyzer import.PNG

Open the (external) CSV file, in this example we use Perfmon CSV file that is collected during the same Login VSI test.

Import xternal data.PNG

Select the metrics you want to import.

Select the metrics.PNG

A new tab is opened with the title External Data (beta)

All metrics are down/up scaled to be in the range of 0-100, by finding the max value and multiplying accordingly. This feature was expanded to be able to hold multiple sources and multiple imports of metrics with ability to create new tabs or merge to the existing external data tabs. This also adds the remaining external data series to moving box, and adds the source column in the data grid view.

The blue steps line in the picture is representing the active sessions. Everything is recalculated to time in seconds, so the X-axis is time in seconds.

External Performance Data Perfmon.PNG

Titles.xml

XML

Description

Screenshot

Section type

This contains the title for the tab opened in the analayzer. For example when I open a ESXtop file I can define in the section type it is a ESXtop file which will be used as title in the tab. This way you know what kind of data is imported

Section type.PNG

Title

Section to define a metric

Title type.png

Displayname

Simple name to identify the selected metric

Display name.png

Columntitle

This should contain the title of the column of the metric you want to import

Columntitle name.png

Multiplier

Here you can multiply the metric you selected 1.0 is the default

Multiplier.png

Description

Screenshot

Open the titles.xml {VSIshare}\_VSI_Binaries\Analyzer\titles.xml. The titles.xml contains the performance data template. Here you can specify the counters you want to show when opening a CSV file

Titles.PNG
Xml file.PNG

Login VSI 4.1 Beta Workloads

New and improved workloads: Two workloads are added; the “Knowledge worker” for 2vCPU desktops and by popular demand the “Office worker” for 1vCPU desktops will be reintroduced. Both workloads are based on the Medium workload introduced in Login VSI 4.0.

The main difference between the “Office” and the “Knowledge worker” is the reduced CPU usage by removing the video’s and the freemind (java) application. The office worker profile is focused on broad Office and Internet Explorer usage and is designed to run within 1vCPU virtual desktops.

Login VSI will recommend the knowledge worker as the standard and is designed to run in 2vCPU desktop to properly support the web video content.

In 4.1 the relative weight between the different workloads is more evenly balanced to better reflect real-life usage. The video library has also been improved, all video formats are now generated from the same source files. The footprint of the Video library is now considerably smaller, which makes the installation and setup of the VSI data/web server easier.


  • Renamed workloads
    • Light <> Task worker
    • Medium <> Knowledge user
    • Heavy <> Power user
  • Rebalanced workloads
    • The relative weight between workloads has a better balance
  • New workloads
    • Office user
      • Requires only 1vCPU
      • Fits in between Task worker and Knowledge user
      • Based on the knowledge user, but less intensive on CPU
    • Knowledge user
      • Completely revised medium 4.0 workload
      • Contains the new 4.1 VSImax timer mechanism
  • Improved VSImax timer mechanism
    • Removed Microsoft word (from the timer, not the workload)
  • Added two new timers
    • 1 specifically for CPU
    • 1 specifically for Disk / Storage performance
  • Improved video library
    • New 240p, 360p, 480p, 720p, 1080p video’s
    • All video formats are generated from the same source files
    • Video Player resizes according to video dimensions


Login VSI 4.1 Beta Timer

Timers All previous versions of Login VSI used Microsoft Office Word within their timer mechanism to calculate the desktops performance. As published in Project VRC whitepaper “Phase 6: Microsoft Office Impact on VDI” the influence of Microsoft Office on the test results is significant. In Login VSI 4.1 Microsoft Word is removed from the timer. This improves the consistency of the test results across different environments. Microsoft Word will still be part of the workloads, just not of the VSI response timer anymore.

The Notepad-File-Print timer (NFP) has been replaced by a timer that measures purely CPU removing the fluctuations that faulty printer settings, a large amount of printers etc. would have on the results. This timer is also a great indicator for bottlenecks, as it’s a 100% CPU dependent it will stay stable when an IO limit is hit therefore indicating where not to look for the bottleneck.

Zip-No-Compression (ZNC) has been replaced by Zip-Low-Compression (ZLC) because it fluctuated between different environments and was dominating the VSImax calculation, especially when storage optimization or anti-virus where part of the test. To prevent caching the zips are now encrypted with a random password.

There are also additional timers added that do not influence the VSImax but are worth a mention.

  • IO-timer, indicates how long it takes to write one random block of 2mb data, this timer is nog included in the VSImax calculation because it can fluctuate greatly but it’s a great bottleneck indicator.
  • UMEM, Show the percentage of used memory per virtual machine

Calculation improvements

  • The base phase is no longer required to calculate the baseline performance of the system, this reduces the complexity of the test and gives more reliable results when for example the power settings have been configured to “balanced”
  • The Analyzer now also allows merging test results to calculate the average over multiple tests.