View source for Login VSI Graphics Framework

'''Important''': Please be aware that this is legacy software. The new iteration is called the [https://www.loginvsi.com/documents/documentation/Login-VSI-Multimedia-Workload.pdf '''Login VSI Multimedia Workload'''], which is designed to really stress the CPU in GPU-enabled VDI/SBC environments.

===Graphics Framework Admin Guide===

This is the Login VSI Graphics Framework Admin Guide. The recommended usage of the Login VSI Professional Graphics Testing Framework is as follows:

* The Professional Graphics Testing Framework relies on Login VSI 4.1, so the first step is to download, install and configure Login VSI 4.1. If you’re not familiar with Login VSI, please refer to the Login VSI installation documentation.  Note, Installing the PRO content library is not needed to use the Professional Graphics Testing Framework.
* Install the Login VSI Professional Graphics Testing Framework into the Login VSI share.
* Validate the GPU-enabled VDI or SBC environment and the Login VSI installation by running one of the graphics workloads, included in the Login VSI Professional Graphics Testing Framework download.
* Create a custom workload, using the provided empty Graphics Framework Workload. This custom workload would contain your own application, models and shading modes.
* Run a test slowly incrementing number of sessions.
* Analyze the results of the test. To get full insight in the performance of your VDI or SBC environment, external data should be gathered and loaded into the Graphics Analyzer.

===Installation===

The Login VSI Professional Graphics Testing Framework relies on the functionality of Login VSI 4.1.

====Download and install Login VSI 4.1====
Download Login VSI from  this [https://www.loginvsi.com/downloads/login-vsi download page]

Install Login VSI 4.1 as described in the [https://www.loginvsi.com/documentation/index.php?title=Login_VSI Login VSI Installation] section.

====Install Login VSI graphics framework====
{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
Start the LoginVSI_Graphics_Framework.exe installer.








|
[[File:GFXI1.png|400px|thumb]]

|-
|
Click the “Next” button.





|

[[File:GFXI2.png|400px|thumb]]
|-
|
Enter the path to your Login VSI share and click the “Next” button.



|
[[File:GFXI3.png|400px|thumb]]

|-
|
Choose “Full installation” and press “Next”.




|
[[File:GFXI4.png|400px|thumb]]

|-
|
Select the icons to be created and press the “Next” button.


|
[[File:GFXI5.png|400px|thumb]]
|-
|
Click the “Install” button to start the installation.


|
[[File:GFXI6.png|400px|thumb]]
|-
|
Wait until the installation is complete.


|
[[File:GFXI7.png|400px|thumb]]
|-
|
Press the “Finish” button to close the installer application.


|
[[File:GFXI8.png|400px|thumb]]
|}

===Running workloads===
The Login VSI Professional Graphics Testing Framework includes two pre-built graphics workloads:

* Autodesk AutoCAD 2014
The AutoCAD 2014 workload has been created in cooperation with NVIDIA for the popular CAD application. This workload is based on the 2014 version of the application and can be used with the AutoCAD 2014 Trial version.

* Unigine Valley Benchmark
Based on the popular engine which allows for OpenGL and DirectX GPU benchmarking.

The Unigine or AutoCAD workload should be used to validate the GPU-enabled VDI or SBC environment and the Login VSI installation before using other applications or modifying the existing workloads. Both the AutoCAD and Unigine can be modified to allow multiple testing methods. Different visualization modes in AutoCAD have different impact on GPU usage and result in different frame rate. This also applies to different models, model complexity will have an impact on the frame rate. The Professional Graphics Testing Framework is created to allow a dynamic way of testing graphics workloads with different models and visualization modes.

The logon rate for graphical workloads should be configured to allow at least one full workload loop to finish. By allowing one full loop to finish will provide best insight in the impact of loading sessions onto the target system. The Unigine workload for example, takes about 5 minutes. Taking the logon and connection time into account, means that a session needs to be started every 420 seconds.

To allow full insight in the performance of the complete environment, external data should be gathered. The graphics analyzer allows importing of CSV files to create a complete overview of both the test data and the host data.

====Running the Unigine workload====
=====Preparing the target machine=====
The Login VSI Professional Graphics Testing Framework makes use of an application called FRAPS. FRAPS allows for framerate detection for DirectX and OpenGL applications. The steps described in the chapter need to be conducted on the target machine.

{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
Download FRAPS from http://www.fraps.com








|
[[File:GFXFRAPS1.png|400px|thumb]]

|-
|
Start the FRAPS installer and press “I Agree”





|

[[File:GFXFRAPS2.png|400px|thumb]]
|-
|
Enter “C:\Program Files\FRAPS” as the destination folder and press “Next”. 

Do not install to the default installation location. This will result in admin-privileges messages during the workload.




|
[[File:GFXFRAPS3.png|400px|thumb]]

|-
|
Press “Install” to start the installation.




|
[[File:GFXFRAPS4.png|400px|thumb]]

|-
|
Wait until the installation is complete.


|
[[File:GFXFRAPS5.png|400px|thumb]]
|-
|
Press “Close” to close the installation.

|
[[File:GFXFRAPS6.png|400px|thumb]]
|}

While Unigine created multiple benchmarks, like Heaven and Tropics, the Unigine workload in the Login VSI Professional Graphics Testing Framework uses the “Valley” benchmark. The workload relies on the Advanced version of the Valley benchmark, since the free version does not allow automation and reporting to CSV files. The software needs to be installed prior to starting the workload.

{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
Purchase and download Unigine Valley Advanced from https://unigine.com/products/valley/








|
[[File:GFXUNI1.png|400px|thumb]]

|-
|
Start the “Unigine_Valley-1.0-Advanced.exe” installer file.





|

[[File:GFXUNI2.png|400px|thumb]]
|-
|
Press “Next” on the Welcome screen.



|
[[File:GFXUNI3.png|400px|thumb]]

|-
|
Click the radio button “I accept the agreement” and click the “Next” button.




|
[[File:GFXUNI4.png|400px|thumb]]

|-
|
Enter the correct user name, organization and serial number.


|
[[File:GFXUNI5.png|400px|thumb]]
|-
|
Leave the installation directory default (C:\Unigine\Valley Benchmark 1.0 Advanced) and press the “Next” button.

|
[[File:GFXUNI6.png|400px|thumb]]
|-
|
Press “Next” in the shortcuts creation screen.

|
[[File:GFXUNI7.png|400px|thumb]]
|-
|
Press “Install” to start the installation.

|
[[File:GFXUNI8.png|400px|thumb]]
|-
|
Wait until the installation is finished.

|
[[File:GFXUNI9.png|400px|thumb]]
|-
|
Press “Finish” to close the installation.

|
[[File:GFXUNI10.png|400px|thumb]]
|}

After FRAPS and Unigine Valley Advanced have been installed, the target machine should be finalized and prepared for implementation. The steps needed to finalize and prepare the machine depend on the infrastructure and are out of scope from this implementation guide.

=====Running the workload=====
As described before, the logon rate per session should be long enough to allow a session to logon and finish one loop of the Unigine workload. One workload loop using the Unigine Valley will take about 5 minutes, taking in account the time it takes to logon a session, the minimum overall logon time should be about 7 minutes (420 seconds).

{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
Configure the Login VSI infrastructure, phase and connection as shown in the “Getting started with Login VSI” video.








|
{{#ev:youtube|UniXxonGQnk}}

|-
|
Open workload > settings in the Login VSI Management Console.





|

[[File:GFXVSI2.png|400px|thumb]]
|-
|
Configure the “Microsoft Office version” to any value.

The Graphics Framework does not use Microsoft Office, but a test can’t be started if no Office version is configured in the Management Console.




|
[[File:GFXVSI3.png|400px|thumb]]

|-
|
Open test setup > scenario in the Login VSI Management Console.




|
[[File:GFXVSI4.png|400px|thumb]]

|-
|
Select “gfx_unigine” as workload.


|
[[File:GFXVSI5.png|400px|thumb]]
|-
|
Configure the phase to launch the target number of sessions.

|
[[File:GFXVSI6.png|400px|thumb]]
|-
|
Configure the “Overall Logon Rate” to be at least 420.

|
[[File:GFXVSI7.png|400px|thumb]]
|-
|
Start a test as described in the Login VSI [[Start your first test]] documentation. [[Start your first test]]

|
[[File:GFXVSI8.png|400px|thumb]]
|-
|
Monitor the session’s progress.

|
[[File:GFXVSI9.png|400px|thumb]]
|-
|
Wait for all sessions to log off. This can be monitored in the Login VSI Management Console dashboard.

|
[[File:GFXVSI10.png|400px|thumb]]
|}

=====Modifying the workload parameters=====
By default, the Unigine workload will use the OpenGL API, no anti-aliasing and will be rendering in the highest quality. These properties can be modified in the workload to change the way the scenes are rendered, changing the load on the CPU and GPU.

Make sure that the Unigine workload file is backed-up before modifying the command line. All the command line options are case-sensitive. To modify the workload parameters, open the workload file via workload > customization in the Login VSI Management Console. Edit the workload and go to line 41, it should display this value:

'''Syntax'''
{| class="wikitable"
|-
|
<nowiki>VSI_ShellExecute("Unigine", "Valley.exe", '-extern_plugin GPUMonitor -extern_define RELEASE,VALLEY_ADV,AUTOMATION,QUALITY_ULTRA -video_app opengl -sound_app openal -project_name Valley -data_path ../ -system_script valley/unigine.cpp -engine_config ../data/valley_1.0.cfg -video_multisample 0 -video_fullscreen 0 -video_mode -1 -frame -1 -duration 0 -type 0 -level 2 -video_width %VSI_CanvasWidth% -video_height %VSI_CanvasHeight% -frames "%TEMP%\VSI\Unigine\UnigineFrames.log" -scores "%TEMP%\VSI\Unigine\UnigineScores.log" -format "$z,$x,$F"', "C:\Unigine\Valley Benchmark 1.0 Advanced\bin") </nowiki>
|}

To eg. change the API to DirectX 11, look up the “-video_app opengl” and change “opengl” into “direct3d11”. The table below contains a list of configuration changes that can be made in the command line of Unigine.

{| class="wikitable"
|-
|
'''Option'''

|
'''Possible values'''

|
'''Description'''

|-
|

-video_app

|

opengl
direct3d11
direct3d9

|

Defines which API to use.
opengl = OpenGL
direct3d11 = DirectX 11
direct3d9 = DirectX 9

|-
|

-extern_define

|

QUALITY_LOW
QUALITY_MEDIUM
QUALITY_HIGH
QUALITY_ULTRA

|

Quality to use when rendering the graphics. Be sure to modify only the quality part in the command line. All other values defined in the –extern_define parameter should be left unchanged.
|-
|

-video_multisample

|

0
1
2
3

|

Anti-aliasing value to use:
0 = No anti-aliasing
1 = 2x
2 = 4x
3 = 8x
|}

====Running the AutoCAD 2014 workload====
{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
Download the AutoCAD 2014 installer.








|
[[File:GFXAC2.png|400px|thumb]]

|-
|
Start the “AutoCAD_2014_English_Win_64bit_dlm.sfx.exe” executable to extract the installation.





|

[[File:GFXAC2.png|400px|thumb]]
|-
|
Leave the default destination folder to “C:\Autodesk” and press “OK” to start the extraction process.




|
[[File:GFXAC3.png|400px|thumb]]

|-
|
Wait until the data has been extracted.




|
[[File:GFXAC4.png|400px|thumb]]

|-
|
The AutoCAD 2014 installation application will be launched automatically. Press the “Install” button.


|
[[File:GFXAC5.png|400px|thumb]]
|-
|
Select the correct “Country or region”, check the “I Accept” checkbox and press the “Next” button.

|
[[File:GFXAC6.png|400px|thumb]]
|-
|
Leave the “Product language” to English and “License type” to “Stand-Alone”. Select the “I want to try this product for 30 days” checkbox and press “Next”.

|
[[File:GFXAC7.png|400px|thumb]]
|-
|
Leave the installation to default, make sure that AutoCAD is installed to “C:\Program Files\Autodesk\”. Press “Install” to start the installation.

|
[[File:GFXAC8.png|400px|thumb]]
|-
|
Wait until AutoCAD has been installed.

|
[[File:GFXAC9.png|400px|thumb]]
|-
|
Press “Finish” to close the installation.

|
[[File:GFXAC10.png|400px|thumb]]
|}

After AutoCAD has been installed, the target machine should be finalized and prepared for implementation. The steps needed to finalize and prepare the machine depend on the infrastructure and are out of scope from this implementation guide.

=====Running the workload=====
As described before, the logon rate per session should be long enough to allow a session to logon and finish one loop of the AutoCAD workload. One workload loop using AutoCAD will take about 6 minutes, taking in account the time it takes to logon a session, the minimum overall logon time should be about 8 minutes (480 seconds).

{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
Configure the Login VSI infrastructure, phase and connection as shown in the “Getting started with Login VSI” video.








|
{{#ev:youtube|UniXxonGQnk}}

|-
|
Open test setup > scenario in the Login VSI Management Console.





|

[[File:GFXVSI4.png|400px|thumb]]
|-
|
Select “gfx_autocad_2014” as workload.




|
[[File:GFXACP2.png|400px|thumb]]

|-
|
Configure the phase to launch the target number of sessions.



|
[[File:GFXACP3.png|400px|thumb]]

|-
|
Configure the “Overall Logon Rate” to be at least 480.


|
[[File:GFXACP4.png|400px|thumb]]
|-
|
Start a test as described in the Login VSI [[Start your first test]] documentation. [[Start your first test]]

|
[[File:GFXACP5.png|400px|thumb]]
|-
|
Monitor the session’s progress.

|
[[File:GFXACP6.png|400px|thumb]]
|-
|
Wait for all sessions to log off. This can be monitored in the Login VSI Management Console dashboard.

|
[[File:GFXACP7.png|400px|thumb]]
|}

=====Modifying the workload parameters=====
By default, the AutoCAD 2014 workload will use one model and 4 shading modes (conceptual, hidden, shaded and shadedwithedges). The AutoCAD 2014 workload allows any number of models to be used with any number of shading modes. This can easily be done by modifying the AutoCAD workload configuration file. This configuration file is located in the VSI Share directory:

* “_VSI_Binaries\\External\NVIDIA\ACADWorkload” 
and is called 
* “AutoCAD2014.ini”


======Additional models======
The AutoCAD 2014 workload contains a list of models, but can be extended or replaced by any AutoCAD model. By default, the Login VSI Professional Graphics Testing Framework contains the following models:

* Aerial.dwg
* ApartmentTower.dwg
* Bed3D.dwg
* Condominium.dwg
* ConferenceRoom.dwg
* SunandSkyDemo.dwg
* WaspEngine.dwg

These models are placed in the: 

* “_VSI_Content\dwg” 
folder of the VSI share and are copied to the target machine in the prepare segment of the AutoCAD workload. To modify the used models in the AutoCAD workload, open the 
* “AutoCAD2014.ini”
file from the 
* “_VSI_Binaries\\External\NVIDIA\ACADWorkload”
directory in the VSI share. The models being used in the workload are defined on line 2:

'''Syntax'''
{| class="wikitable"
|-
|
<nowiki> Model=Data\AutoCAD 2014\ApartmentTower.dwg </nowiki>
|}


The used models can be updated by adding more models using a comma as separator. Using multiple models would, for example, be configured like this:

'''Syntax'''
{| class="wikitable"
|-
|
<nowiki> Model=Data\AutoCAD 2014\ApartmentTower.dwg,Data\AutoCAD 2014\WaspEngine.dwg </nowiki>
|}

Adding additional models can be done by copying the model files to the 
* “_VSI_Content\dwg” 
folder in the VSI share and updating the configuration file as described before. The files need to be copied to the target machine, which can be done in the workload file. Open the workload file via workload > customization in the Login VSI Management Console. Edit the workload and go to line 43, add a new line to the workload for every model that needs to be configured. Copying a model in the workload is done as follows:

'''Syntax'''
{| class="wikitable"
|-
|
<nowiki> VSI_File_Copy("Workload", "%VSI_DataLocation%\dwg\<ModelName>.dwg", "%TEMP%\VSI\NVIDIA_acadWorkload\Data\AutoCAD 2014") </nowiki>
|}

Note that updating the model count has an impact on the workload loop length. More complex models will cause the workload loop time to increase. This means that the overall logon rate should be changed accordingly.

======Additional visualization modes======
By default, the AutoCAD workload will use 4 visualization modes. This behavior can be changed by modifying the 
* “AutoCAD2014.ini” 
configuration file which is located in the 
* “_VSI_Binaries\\External\NVIDIA\ACADWorkload” 
directory of the VSI share. The possible visualization modes are as follows:

* Conceptual
* Hidden
* Realistic
* Shaded
* ShadedWithEdges
* ShadesOfGray
* Sketchy
* Wireframe
* XRay

To add or remove visualization styles, open the 
* “AutoCAD2014.ini” 
configuration file and go to line 3:

'''Syntax'''
{| class="wikitable"
|-
|
<nowiki> VisualStyle=Conceptual,Hidden,Shaded,ShadedWithEdges </nowiki>
|}

The different visualization styles are separated by a comma. To add the “Realistic” style and remove the “Hidden” style, for example, the configuration should be:

'''Syntax'''
{| class="wikitable"
|-
|
<nowiki> VisualStyle=Conceptual,Shaded,ShadedWithEdges,Realistic </nowiki>
|}

Note that updating the visualization style has an impact on the workload loop length. This means that the overall logon rate should be changed accordingly.

===Customizing Workloads===

====Creating a custom Graphics Workload====
The Login VSI Graphics Framework contains two per-built workloads, the AutoCAD 2014 workload and the Unigine Valley workload. These workloads can be used to measure application performance, but in most cases, customers will need to implement their own graphical applications.

By using the empty Graphics Framework workload as a base, custom applications like Catia, 3DS Max or any other graphics application can easily be added to a custom workload. When creating custom workloads, keep in mind that framerate is measured by the Login VSI Graphics Framework on multiple levels:

*Framerate reported by the application itself, if possible
*Framerate of the application measured inside the guest VM by the FRAPS tool
*Framerate forwarded over the remoting protocol by the guest VM to the endpoint device

The preferred method of measuring framerate inside the guest VM is by the application itself. Applications like AutoCAD can report on measured framerate by itself, but not all applications support this method. If the application does not report framerate, the framerate can be measured by using the FRAPS tool.

Before building a custom workload, think of the scenario you want to test using your application. The scenario should resemble actions which are also done real-world. Think of the actions your end-users would do, for example opening a model, spinning a model, replacing a component, etc. After that, check how to application can be automated to do these actions. The preferred method of automation is by using some kind of scripting, for example an automation API, or by using keyboard shortcuts.

First step in creating a custom workload, is copying the Graphics Framework template workload. This will be used as base for the custom workload.

====Login VSI Graphics Framework Workload Commands====

For more information about the commands used in the Login VSI Graphics Framework Workload commands please [http://www.loginvsi.com/documentation/Login_VSI_Workload_Language_Reference_Guide#VSI_Graphics_Framework_Functions click here.]

====Copying the Graphics Framework Template====
{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
Start the Login VSI Management Console




|
[[File:GFX-Customizing-1.png|400px|thumb]]

|-
|
Go to Workload > Customization


|

[[File:GFX-Customizing-2.png|400px|thumb]]
|-
|
Select the “Graphics_Framework” workload file


|
[[File:GFX-Customizing-3.png|400px|thumb]]

|-
|
Press the “copy” button


|
[[File:GFX-Customizing-4.png|400px|thumb]]

|-
|
Select the “Graphics_Framework – Copy.txt” workload


|
[[File:GFX-Customizing-5.png|400px|thumb]]

|-
|
Press “rename”

|
[[File:GFX-Customizing-6.png|400px|thumb]]

|-
|
Fill in a new name, eg. “Graphics_MyApp“ and press on the “rename workload” button

|
[[File:GFX-Customizing-7.png|400px|thumb]]

|-
|
Select the newly created workload and press the “edit” button


|
[[File:GFX-Customizing-8.png|400px|thumb]]

|-
|
The workload editor window opens, refer to the next part of the documentation for pointers on creating a custom graphics workload


|
[[File:GFX-Customizing-9.png|400px|thumb]]
|}

====Modifying the template workload====
Once the workload template has been copied, the custom application can be added to the workload file. The template workload contains markers, with a prefix of '''DOC_ID''', which can be found in this manual.

=====Preparing the application=====
'''[DOC_ID1]''' Before the workload starts, some applications require some preparation. For example starting the application and closing it to be sure that the application configuration has been created in the user’s profile and registry. Actions to clean up registry keys or files before the application starts or copying over files that are needed by the application can be inserted here. These actions are executed only once in the entire workload, before the actual workload starts.

Typically, the following workload actions are inserted at this point:

*App_Close
*App_Focus
*VSI_DirCreate
*VSI_DirCopy
*VSI_DirDelete
*VSI_File_Copy
*VSI_Killprocess
*VSI_RegDelete
*VSI_RegWrite
*VSI_RegRead
*VSI_ShellExecute

'''Example workload actions:'''

{| class="wikitable"
|-
|
<nowiki># Clean up profile</nowiki><br>
<nowiki>VSI_DirDelete(“CleanUp”, “%APPDATA%\MyApp”)</nowiki><br>
<nowiki>VSI_RegDelete(“CleanUp”, “HKEY_CURRENT_USER\SOFTWARE\MyApp”)</nowiki><br>

<nowiki># Start and close the application to create a profile</nowiki><br>
<nowiki>VSI_ShellExecute(“PrepareApp”, “myApp.exe”, “-start”, “%PROGRAMFILES%\MyApp”)</nowiki><br>
<nowiki>App_Focus(“PrepareApp”, “Title”, “My App Title”)</nowiki><br>
<nowiki>App_Close(“PrepareApp”, “Title”, “My App Title”)</nowiki><br>
|}

=====Cleaning up data=====
'''[DOC_ID2]''' Some applications leave unwanted or unneeded data on the target machine once the application has finished its actions. Any cleanup needs to be done every time a workload loop has finished, can be done here. These actions are executed every time a new loop is started.

Typically, the following workload actions are inserted at this point:

*VSI_DirDelete
*VSI_RegDelete

'''Example workload actions:'''

{| class="wikitable"
|-
|
<nowiki># Clean up profile</nowiki><br>
<nowiki>VSI_DirDelete(“WorkloadCleanUp”, “%TEMP%\MyApp”)</nowiki><br>
<nowiki>VSI_RegDelete(“WorkloadCleanUp”, “HKEY_CURRENT_USER\SOFTWARE\MyApp\TempData”)</nowiki><br>
|}

=====Running the application=====
'''[DOC_ID3]''' Once all application data has been cleaned up, the application can be started. Like stated before, the preferred method of using applications in the Login VSI Graphics Framework is by making use of automation, like the applications API or scripting methods. If the application does not provide an API or scripting methods, keystrokes should be simulated to do the required actions in the application. Alternatively, if the application supports loading of macros, a macro can be used to simulate application usage.

Typically, the following workload actions are inserted at this point:

*App_Start
*VSI_ShellExecute

The application started can either be the application itself, or a script or API wrapper which handles all actions that should be done in the application.

'''Example workload actions:'''
{| class="wikitable"
|-
|
<nowiki># Start the application</nowiki><br>
<nowiki>VSI_ShellExecute(“WorkloadApp”, “myApp.exe”, “”, “%PROGRAMFILES%\MyApp”)</nowiki><br>
|}

=====Application actions and waiting until the application finished=====
'''[DOC_ID4]''' Once the application has been started, the Login VSI data collector is started, as well as FRAPS framerate capturing. These 2 actions are already inserted into the template workload. In this part of the workload, there are 2 options:

1.	If an external script or API wrapper (compiled executable) is being used to do the application actions, the workload should wait until the actions are done and the script or API wrapper is closed.
2.	If the application does not have an API or support scripting, the actions done in the application should be inserted in the workload at this point. After the actions are done, preferably by using only keystrokes, the application should be closed.

======Using an external script======
When an external script or API wrapper is used, the workload has to wait until the application closes. This can be done by using one of the following workload actions:

*VSI_Wait_Content
*VSI_Wait_AppClose
*VSI_WaitWindow
*VSI_WaitProcess

'''Example workload actions:'''
{| class="wikitable"
|-
|
<nowiki># Wait for the application to close</nowiki><br>
<nowiki>VSI_Wait_AppClose(“WorkloadApp”, “Title”, “My App Name”)</nowiki><br>
|}

'''Example 2:'''
{| class="wikitable"
|-
|
<nowiki># Wait for content in a log file</nowiki><br>
<nowiki>VSI_Wait_Content(“WorkloadApp”, “%TEMP%\MyAppLogfile.txt”, “Application Finished”)</nowiki><br>
|}

=====Application actions=====
If the application does not have an API or support some kind of scripting, the actions done inside the application should be done here. The preferred method of doing actions inside an application is by using keystrokes. For example, playing 

Automating an application by using keystrokes can be done using the following workload actions:
*App_Focus
*VSI_Mouse_Click (not preferred)
*VSI_Mouse_Position (not preferred)
*VSI_Type_Fixed

Besides automating the application, certain workload actions need to be added to wait for the application to be ready. The following workload actions can be used:
*VSI_Wait_Content
*VSI_Wait_AppClose
*VSI_WaitWindow
*VSI_WaitProcess

'''Example workload actions:'''
{| class="wikitable"
|-
|
<nowiki># Focus on the application</nowiki><br>
<nowiki>App_Focus(“WorkloadApp”, ”Title”, ”My App Name”)</nowiki><br>
<nowiki># Press CTRL-O to open a model</nowiki><br>
<nowiki>VSI_Type_Fixed(“WorkloadApp”, “{ctrldown}o{ctrlup}”)</nowiki><br>
<nowiki>App_Focus(“WorkloadApp”, ”Title”, ”Open Window”)</nowiki><br>
<nowiki>VSI_Type_Fixed(“WorkloadApp”, “C:\Data\MyModel.dwg{enter}”)</nowiki><br>
<nowiki># Sleep 5 seconds</nowiki><br>
<nowiki>VSI_Sleep(5)</nowiki><br>
<nowiki># Press F5 to animate and wait until the logfile contains a specific text</nowiki><br>
<nowiki>VSI_Type_Fixed(“WorkloadApp”, “{f5}”)</nowiki><br>
<nowiki>VSI_Wait_Content(“WorkloadApp”, “%TEMP%\MyAppLogfile.txt”, “Application Finished”)</nowiki><br>
|}

======Parsing external configuration files to gather performance data======
'''[DOC_ID5]''' If an application can output CSV or text files with performance data, for example application framerate, the CSV can be interpreted by the Login VSI Workload Engine and translated to Login VSI logfiles. By default, the template workload file contains parsing of Login VSI Data Collector and FRAPS logfiles. To read more about parsing performance data, refer to the “Parsing Graphical Performance Data” part of this manual.

At this point, the following workload action would be used:
*VSI_ParseGFXData

'''Example workload actions:'''
{| class="wikitable"
|-
|
<nowiki># Parse my custom application CSV output file</nowiki><br>
<nowiki>VSI_ParseGFXData(“ParseMyApp”, “MyAppConfig.ini”, “GFXAppData”, 0, “AppTimeMarker”)</nowiki><br>
|}

======Cleaning up or copying data after the workload is finished======
'''[DOC_ID6]''' At the end of the workload, just before the test user will log off, there is an option to do additional cleanup or gathering of data. This any workload actions executed at this point of the workload, will only be done once in the entire workload. If the application requires any cleaning up, like releasing a license, or remove a file from the network, this is the location in the workload to execute these actions. Copying performance data which has been gathered during the execution of the workload, by for example PerfMon, can be copied at this point.

Typically, the following workload actions are inserted at this point:

*VSI_File_Copy
*VSI_File_Delete
*VSI_RegDelete
*VSI_DirCopy
*VSI_Dir_Copy_Wait

'''Example workload actions:'''
{| class="wikitable"
|-
|
<nowiki># Stop the PerfMon collection and copy it to a share</nowiki><br>
<nowiki>VSI_ShellExecute(“FinalizeApp”, “logman.exe”, “stop myCollector”, “C:\Windows\System32”)</nowiki><br>
<nowiki>VSI_File_Copy(“FinalizeApp”, “C:\PerfData\Admin\MyLog\myCollector.blg”, “\\SERVER\MyShare\%COMPUTERNAME%_Perf.blg”)</nowiki><br>
|}

=====Defining a time marker=====
By default, the Login VSI workloads use the time and date in the workload when a certain workload action is executed. For example, the VSI_ParseGFXData() function by default uses the time and date when the action is executed. However, when data is sampled over longer periods of time, the date and time when the data is parsed may not be the right value. For these kinds of scenarios, the VSI_TimeMarker() function can be used.

The following time line will be used as example:

[[File:GraphicsData-TimeMarker.png|400px|thumb|center]]

At 10:03 the framerate measurement is started, this is typically the same time as the application is started. At 10:08 measurement is stopped and at 10:09 the data is parsed. Normally, when you use the VSI_ParseGFXData() function, the timestamp written to the Login VSI log files starts at 10:09 because that’s the time the function is executed. However, in this case, the timestamp should start at 10:03, since that’s the starting time of the framerate measurement. The VSI_TimeMarker() function is added for this purpose. The VSI_TimeMarker() function can be executed just before framerate measurement starts at 10:03, when the time marker ID is used in the VSI_ParseGFXData() function, the timestamp inserted into the Login VSI log files will be 10:03 instead of 10:09.

=====Parsing Graphical Performance Data=====
Applications like FRAPS and Unigine can output log files which contain framerate data of the application itself. The preferred method of gathering framerate data of an application is by letting the application itself output this data. By using the VSI_ParseGFXData(), this data can be interpreted and translated to Login VSI compatible log files.

The VSI_ParseGFXData() function uses configuration files to translate the data found in the application’s log files. To allow parsing of log files, the configuration file should be created first. The file is be stored in the “_VSI_Configuration\_ExternalDataConfig” folder of the Login VSI share. A typical configuration file looks like this:

{| class="wikitable"
|-
|
<nowiki>[config]</nowiki><br>
<nowiki>DataPath=%TEMP%</nowiki><br>
<nowiki>Files=1</nowiki><br>

<nowiki>[file_1]</nowiki><br>
<nowiki>FileName=CPUTotal.csv</nowiki><br>
<nowiki>UseTimeMarker=1</nowiki><br>
<nowiki>RowTimeIncrease=5</nowiki><br>
<nowiki>Type=CSV</nowiki><br>
<nowiki>Delimiter=,</nowiki><br>
<nowiki>Fields=2</nowiki><br>
<nowiki>SkipFirstRow=1</nowiki><br>
<nowiki>LogName=Data</nowiki><br>

<nowiki>[file_1_field_1]</nowiki><br>
<nowiki>Name=CPUload</nowiki><br>
<nowiki>Include=1</nowiki><br>
<nowiki>RoundData=1</nowiki><br>

<nowiki>[file_1_field_2]</nowiki><br>
<nowiki>Name=FreeMemoryMB</nowiki><br>
<nowiki>Include=1</nowiki><br>
<nowiki>RoundData=1</nowiki><br>
|}

The first section of the configuration file, always defined by the [config] directive, is the basic configuration. This section contains only 2 fields:

1.	DataPath
The path to search for the generated log files

2.	Files
The number of log files that should be parsed

Next section defines the configuration for each log file to be parsed. This configuration is always prefixed by the “file_” directive. If only one file should be parsed, there should only be a [file_1] section. If, for example, two files should be parsed, there should be two sections, called [file_1] and [file_2]. The number of files is defined in the [config] section as described before. The [file_*] section contains the following configuration fields:

1.	FileName
The name of the log file to search for, this field can contain wildcards. If multiple files are found, for example if a wildcard is used, only the first file will be parsed.

2.	UseTimeMarker
The value of this field can either be 1 or 0. It defines if a time marker should be used as starting point for the found performance data. This field should contain a value of “1” if the VSI_TimeMarker() function is used in the workload. If this field contains “0”, the current date and time are used as starting time.

3.	RowTimeIncrease
If multiple rows are found, this fields defines how many seconds should be added to the starting time. For example, if this field contains “5”, the starting time (either defined by the time marker or the current time) is 11:25:10 and the log file contains 3 rows of data, the data will log the times 11:25:10, 11:25:15 and 11:25:20.

4.	Type
This defines what kind of data is stored in the log files. At this moment, the only supported format is CSV.

5.	Delimiter
If the log file contains multiple data fields, this configuration defines what the delimiter character is. Each row in the log file will be split on this character.

6.	Fields
The number of columns that are parsed. This is related to the Delimiter configuration field. If a log file for example contains 10 fields, but you’re using only field 2,4 and 5, the number of fields defined in this configuration should be 5. The “fields” setting will start counting from the first field in the log file, even if it should not be included in the data translated to Login VSI log files.

7.	SkipFirstRow
This defines if the first row in the log file should be skipped. This should typically happen when your log file contains column names in the first row.

8.	LogName
This is the name that either will be prefixed in front of the fields written in the Login VSI log files if log files should be combined or the suffix for the log directory that will be created in the Login VSI share if the log files should not be combined. The definition of combined log files will be covered below.


For each file, each field is defined next in the configuration file. The field configuration section is marked as [file_*_field_*]. If, for example, there is one file, which contains two fields, the sections [file_1_field_1] and [file_1_field_2] should be defined. The number of fields is described in item 6 above. Each field contains the following settings:

1.	Name
The name that will be used for this field in the Login VSI log file. If log files will be combined, the field will be prefixed by the LogName, defined in item 8 above.

2.	Include
This setting can either contain 1 or 0. This field will only be written to the Login VSI log file if this settings contains a value of “1”. 

3.	RoundData
This setting can either contain 1 or 0. If the settings contains a value of “1”, the value found in this field will be rounded towards a whole number.

4.	IsHeader
If a log file contains the header names in the first column instead of the first row, this will indicate that the specified column will be used for header names.

The image below shows how a log file is translated to a Login VSI log file using the settings described above.

[[File:GraphicsData-Parsing.png|400px|thumb|center]]

Once the configuration file has been created, it can be used by the VSI_ParseGFXData() function. The function takes the following parameters:
{| class="wikitable"
|-
|
<nowiki>VSI_ParseGFXData(</nowiki><br>
<nowiki>“Log name”,</nowiki><br>
<nowiki>“Config File”,</nowiki><br>
<nowiki>[“Log Directory”,]</nowiki><br>
<nowiki>[Combine Results,]</nowiki><br>
<nowiki>[MarkerID]</nowiki><br>
<nowiki>)</nowiki><br>
|}

*Log name
This is the name that will be used when debug logging is enabled. Any name can be chosen as log name.

*Config File
The name of the configuration file, created as described above. The configuration file needs to be stored in the “_VSI_Configuration\_ExternalDataConfig” directory of the Login VSI share.

*Log Directory (optional)
The directory in the Login VSI results folder where the parsed results will be written to. This must be a unique name, unless the “Combine Results” argument has a value of “1”.

*Combine Results (optional)
This argument contains either 1 or 0. When results are combined, the various log files are merged into one Login VSI log file. Combining results is done according to the “Log Directory” argument. Parsed configuration files with the same log directory will be combined.

*MarkerID (optional)
If the VSI_TimeMarker() function is used in the workload, this defines which time marker to use.

If the configuration file is correct, the translated data is visualized in the Login VSI Graphics Analyzer. Each “Log Directory” will get its own tab once the test data is analyzed. To get help on starting a test in Login VSI, please refer to the Login VSI Documentation.

===Analyzing results===
Once the test has finished, the data can be analyzed with the Graphics Analyzer. Because of the dynamic nature of the Professional Graphics Testing Framework, tests can’t be analyzed using the Login VSI Analyzer. The Graphics Analyzer is installed separately from the regular Login VSI Analyzer and can be started either using the 
*“Login VSI Graphics Analyzer” 
shortcut in the VSI share, or using starting the executable 
*“VSI GFX Analyzer.exe” 
directly from the “_VSI_Binaries\GFX_Analyzer” directory in the VSI share.

=====Importing test results=====
{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
Start the “Login VSI Graphics Analyzer” shortcut, located in the VSI share.

Alternatively, the Graphics Analyzer can be started by launching the “VSI GFX Analyzer.exe” from the “_VSI_Binaries\GFX_Analyzer” directory in the VSI share.









|
[[File:GFXIA1.png|400px|thumb]]

|-
|
Once the Graphics Analyzer start, fill in the VSI share and press enter.





|

[[File:GFXIA2.png|400px|thumb]]
|-
|
Only tests which used the Professional Graphics Testing Framework will be displayed.

Select the test that needs to be analyzed and press “Open”.






|
[[File:GFXIA3.png|400px|thumb]]

|-
|
The data will be analyzed and the charts will be shown.
|
[[File:GFXIA4.png|400px|thumb]]
|}

=====Interpreting results=====
The concept behind the Graphics Analyzer is different from the regular Login VSI Analyzer. While the regular analyzer will output a maximum number of sessions for the conducted test, the graphics analyzer will only output charts and will not define a maximum session count. The charts shown will contain data about frames per second (FPS) and performance data like CPU utilization and memory usage.

With regular VSI tests, a lower VSImax score means better performance, however, tests conducted with the Graphics Framework a higher frame rate means better performance. The framework can report frame rate on different levels:

1.	Application frame rate
2.	Frame rate measured by FRAPS
3.	Frame rate forwarded by the protocol

When the built-in workloads are used, all tabs in the analyzer beginning with “GFX” will contain frame rate (FPS) on the Y-axis, while the session count is displayed on the X-axis. For each separate data line, the minimum (_Low), average (_Median) and maximum (_High) will be shown for the measured data per session count.

All tabs not beginning with “GFX” will contain execution time on the X-axis, while the data displayed on the Y-axis can differ per tab. This is the result of the dynamic nature of the Graphics Analyzer, any data imported in the correct format will be displayed in the analyzer as a separate tab.

======Unigine workload data======
{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
'''GFX FRAPSFPS'''

The minimum, average and maximum frame rate, measured by FRAPS. For each measurement, the minimum (_Low), average (_Median) and maximum (_High) per session count is calculated.









|
[[File:GFXAna1.png|400px|thumb]]

|-
|
'''GFX ProtcolFPS'''

Minimum, average and maximum frame rate forwarded by the protocol. This is protocol-agnostic, in case of XenDesktop, the HDX frame rate is measured, in case of VMware View, the PCoIP frame rate is measured. For each measurement, the minimum (_Low), average (_Median) and maximum (_High) per session count is calculated.






|

[[File:GFXAna2.png|400px|thumb]]
|-
|
'''GFX UnigineFPS'''

The minimum, average and maximum frame rate, measured by the Unigine Valley benchmark. For each measurement, the minimum (_Low), average (_Median) and maximum (_High) per session count is calculated.







|
[[File:GFXAna3.png|400px|thumb]]

|-
|
'''FRAPSFPS'''

The frame rate measured by FRAPS over the complete workload, for each single session. The frame rate is displayed over time instead of session count.
A “session stair” is displayed as a dotted line and depicts the logon of sessions in relation to test execution time.




|
[[File:GFXAna4.png|400px|thumb]]

|-
|
'''PerformanceData'''

The CPU utilization and memory usage measured in the target machine for each single session. The performance data is displayed over time instead of session count.
A “session stair” is displayed as a dotted line and depicts the logon of sessions in relation to test execution time.



|
[[File:GFXAna5.png|400px|thumb]]
|-
|
'''ProtocolFPS'''

The frame rate measured on protocol level (protocol agnostic) for each single session. The frame rate is displayed of time instead of session count.
A “session stair” is displayed as a dotted line and depicts the logon of sessions in relation to test execution time.


|
[[File:GFXAna6.png|400px|thumb]]
|}

======Autocad 2014 workload results======
{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
'''GFX ApartmentTower_ConceptualFPS'''

The frame rate measured by AutoCAD for the ApartmentTower model, in Conceptual visualization mode. For each measurement, the minimum (_Low), average (_Median) and maximum (_High) per session count is calculated. 










|
[[File:GFXAu1.png|400px|thumb]]

|-
|
'''GFX ApartmentTower_HiddenFPS'''

The frame rate measured by AutoCAD for the ApartmentTower model, in Hidden visualization mode. For each measurement, the minimum (_Low), average (_Median) and maximum (_High) per session count is calculated.






|

[[File:GFXAu2.png|400px|thumb]]
|-
|
'''GFX ApartmentTower_ShadedwithedgesFPS'''

The frame rate measured by AutoCAD for the ApartmentTower model, in ShadedWithEdges visualization mode. For each measurement, the minimum (_Low), average (_Median) and maximum (_High) per session count is calculated.




|
[[File:GFXAu3.png|400px|thumb]]

|-
|
'''GFX ProtocolFPS'''

Minimum, average and maximum frame rate forwarded by the protocol. This is protocol-agnostic, in case of XenDesktop, the HDX frame rate is measured, in case of VMware View, the PCoIP frame rate is measured. For each measurement, the minimum (_Low), average (_Median) and maximum (_High) per session count is calculated.



|
[[File:GFXAu4.png|400px|thumb]]
|-
|
'''LoadFileTime'''

The time it takes for each model to be loaded for each visualization mode, per session. The performance data is displayed over time instead of session count.
A “session stair” is displayed as a dotted line and depicts the logon of sessions in relation to test execution time.


|
[[File:GFXAu5.png|400px|thumb]]
|-
|
'''PerformanceData'''

The CPU utilization and memory usage measured in the target machine for each single session. The performance data is displayed over time instead of session count.
A “session stair” is displayed as a dotted line and depicts the logon of sessions in relation to test execution time.


|
[[File:GFXAu6.png|400px|thumb]]
|-
|
'''ProtocolFPS'''

The frame rate measured on protocol level (protocol agnostic) for each single session. The frame rate is displayed of time instead of session count.
A “session stair” is displayed as a dotted line and depicts the logon of sessions in relation to test execution time.
|
[[File:GFXAu7.png|400px|thumb]]
|}

=====Importing external data=====
To get full insight in the performance of the environment, it is recommended to gather performance data on different levels, like host CPU utilization or host GPU utilization. The Graphics Analyzer can import CSV data that has been gathered on eg. the virtualization host using ESXtop or RRD2CSV.

{| class="wikitable"
|-
|
'''Description'''

|
'''Screenshot'''

|-
|
Open the test in the Graphics Analyzer.









|
[[File:GFXAna1.png|400px|thumb]]

|-
|
Press File > Import External Data





|

[[File:GFXEI2.png|400px|thumb]]
|-
|
Select the CSV file which contains the external data and press “Open”






|
[[File:GFXEI3.png|400px|thumb]]

|-
|
Select the columns for which a chart should be drawn and press “Next”.



|
[[File:GFXEI4.png|400px|thumb]]

|-
|
Select the existing chart which should be merged with the external data.

To create an empty chart only for the external data, choose “New chart”.

Press the “Next” button.



|
[[File:GFXEI5.png|400px|thumb]]
|-
|
Enter a name which will appear as the tab name and press “Finish”.

|
[[File:GFXEI6.png|400px|thumb]]
|-
|
The external data is visualized as a new chart in the Graphics Analyzer. A multiplier is applied to the data to show the chart between a range of 0 to 100. The applied multiplier is show in the legend.

|
[[File:GFXEI7.png|400px|thumb]]
|}

[[Category:Login VSI]]