VisageTrackerUnityDemo sample project

The VisageTrackerUnityDemo sample project demonstrates the integration of visage|SDK with Unity game engine and is aimed at developers starting to use face tracking functionality of visage|SDK in Unity game engine and development tool.
The application needs to be built from the Unity project which is located in www/Samples/VisageTrackerUnityDemo. Detailed instructions for building and running the project are available in sections Building the project and Running the project. If you wish to integrate visage|SDK in your own Unity project, instructions can be found here.

The sample project shows how face tracking is used to put virtual objects (glasses) on multiple faces in Unity, and how to access the full 3D face model of the faces. The sample also shows the use of face analysis by displaying age, gender and emotion estimation results for each tracked face. Developers may use parts of this project to speed up their own development in Unity using visage|SDK or use this application as a starting point for their own projects.

Building the project

The files related to the VisageTrackerUnityDemo sample project are located in the www/Samples/VisageTrackerUnityDemo subfolder of the visage|SDK folder.

The application was built and tested with version 2020.3.21f1.

NOTE:

The plugin should be modified if it is used in older versions of Unity. More information can be found in the VisageTrackerUnityPlugin and VisageAnalyserUnityPlugin documentation.

Depending on the Unity version that is used, multiple error messages related to project dependencies might appear in the console. Unity offers several possible solutions for this issue and more information can be found here.

To build the application follow these steps:

Open VisageTrackerUnityDemo project in Unity (www/Samples/VisageTrackerUnityDemo).
Add license key to the application (more information can be found in Licensing section of the documentation).
Select and open the Main scene, located in Project's Assets.
Please make sure the Analyser.cs script is attached to the Anaylser object in the Main scene.
Build the application by selecting File->Build settings... and under Platform choose WebGL.
Check the Main scene in window Scenes In Build. If the Main scene is not listed, press Add Open Scenes button.
Press the Build button to generate the project, a dialog window will appear. In this window, locate and set a destination folder to www/Samples/VisageTrackerUnityDemo/Build (the Build folder must first be created).

WebGL build generates 3 folders:

Build
StreamingAssets
TemplateData

and html file:

index.html

Before running:

copy visageSDK.js, visageSDK.wasm, visageSDK.data, visageAnalysisData.js and visageAnalysisData.data from the www/lib folder to the same folder as index.html file (www/Samples/VisageTrackerUnity/Build).

Modify index.html output file to expose unityInstance to the global variable unityGame:


	<body>
		<script>
		...
		var unityGame;
		var script = document.createElement("script");
		script.src = loaderUrl;
		script.onload = () => {
		  createUnityInstance(canvas, config, (progress) => {
			progressBarFull.style.width = 100 * progress + "%";
		  }).then((unityInstance) => {
			loadingBar.style.display = "none";
			unityGame = unityInstance;
			fullscreenButton.onclick = () => {
			  unityInstance.SetFullscreen(1);
			};
		  }).catch((message) => {
			alert(message);
		  });
		};
		document.body.appendChild(script);
		</script>
	</body>

Running the project

The WebGL build output (Samples/VisageTrackerUnity/Build) must be installed on a web server in order to run (it can not run from a local disk).

To install the application on your own web server, upload folder Samples/VisageTrackerUnityDemo/Build to your web server. To run the application access the Samples/VisageTrackerUnityDemo/index.html page via a web browser.
For a quick trial from Visage Technologies web server, click here and choose Visage Tracker Unity.

The application must be licensed in order to function. Licensing is based on License Key Files. To obtain the License Key File:

Contact your Visage Technologies contact person in order to obtain a license key.
Copy the obtained license key file into your application folder (e.g. Samples/VisageTrackerUnityDemo/Assets/StreamingAssets/Visage Tracker).
Set the licenseString member of the LicenseString class (script LicenseString.cs) so it contains the name of the license key file.
Preload license key file to the file system in Awake() function using _preloadFile function (see _preloadFile).

Send the path of the license file to _initializeLicense function (see _initializeLicense).
Example from Tracker.cs (www/Sample/VisageTrackerUnityDemo/Assets/Scripts) script:



	VisageTrackerNative._preloadFile(Application.streamingAssetsPath + "/Visage Tracker/" + LicenseString.licenseString);
	
	licenseFilePath = "";
	
	...

	VisageTrackerNative._initializeLicense(licenseFilePath + LicenseString.licenseString);

For further information please read licensing section of the documentation.

Using the sample application

Running the sample requires WebGL 2.0 compatible browsers.
Allow the application to access your camera by choosing "Allow" in the browser's pop-up bar.
Tracking and face analysis will start automatically when a frontal face is found in the camera frame.
Optional: Play/Pause in the lower right corner
Optional: Switch effect in the lower right corner

Implementation overview

The project consists of the main scene with Unity’s GameObjects that provide different functionalities. A GameObject is the main building block of a scene in Unity. It consists of different Components and can parent other GameObjects. Manipulation of GameObjects is done with Scripts. It is recommended for developers to get familiar with Unity basics before continuing. The integration with visage|SDK is done through a plugin (VisageTrackerUnityPlugin and VisageAnalyserUnityPlugin) that wraps native code calls to visage|SDK and provides functions that can be called from Unity scripts.

Tracker GameObject

The core of the example is the Tracker GameObject. It consists of different components attached to it. The main component of the Tracker GameObject is the Tracker script component that handles the communication with the tracker (starts it and gets results from it). The behaviour of the script can be modified by changing its properties and the script code. Other properties can be added as well. The existing Tracker script properties of interest are as follows:

MAX_FACES parameter: a parameter that determines how many faces in the frame will be tracked
Controllable Objects Glasses: list of Glasses Model GameObjects that will be transformed automatically while the face tracking is performed
Controllable Objects Tiger: list of tiger textures that will be transformed while the face tracking is performed after the button effect is pressed
Config File: filename of the configuration file that will be used by the tracker

Analyser GameObject

Analyser GameObject is another important object that relies on the functionality of the Tracker object, i.e. Tracker script component. The behaviour of this object is defined in the Analyser script. The existing Analyser script properties of interest are:

Analysis Data Element: prefab attached to the Analyser GameObject which contains the components needed to display information about the person's age, gender and emotions
Analysis list: list of Analysis Data Elements instances for each tracked face that will be transformed while the face tracking is performed

Cameras

There are two cameras in the scene, one for the 3D scene (Main Camera) and the other for video display (Video Camera). The main camera "Field of View" is automatically set according to the Tracker focus (camera_focus parameter in the configuration file) and input image aspect by the Tracker script. The Video Camera renders objects in the Video layer by using orthographic projection.

Video object

Video GameObject contains Video Plane that is used for displaying video and is automatically scaled to input frame aspect ratio by the VideoController script attached to it.

Glasses object

The rotation and translation information from the tracker is applied to the Glasses object as they are in the Tracker script Controllable Objects Glasses list. This enables overlaying virtual objects to the tracked face that transform correctly with it. Other custom objects can be used instead of the glasses object. For the correct size and positioning of the glasses objects and also on how to achieve occlusion refer to the Animation & AR Modeling Guide .

VisageUnityPlugin

The integration of visage|SDK face tracking with Unity is done through a plugin that wraps native code calls and provides functions that can be called from Unity scripts. The provided plugin is divided into two files: VisageTrackerUnityPlugin.jslib and VisageAnalyserUnityPlugin.jslib and can be used as-is. For more information about the plugin including its source code, see the documentation of VisageTrackerUnityPlugin and VisageAnalyserUnityPlugin.

Browser compatibility

Visage Tracker Unity Demo sample is tested on the recent browsers on Windows, Mac, Android and iPhone platforms. Results are represented in the following table:


96	91	81	88	14
95	93	64	--	--
--	--	--	--	14