Visage Technologies strives to minimize changes in API and configuration files when releasing new versions of the SDK. The inevitable changes are listed here together with specific instructions for developers who have existing applications built with older versions.
This section covers all changes introduced in all releases since 9.0.
For detailed description of these changes consult VisageTracker Configuration Manual, section 2.1. for additional information.
VisageConfiguration API
Introducing new functions for getting and settingenable_smoothing
parameter:
VisageConfiguration.getEnableSmoothing();
VisageConfiguration.setEnableSmoothing(var enable_smoothing);
Following data files in www/lib folder updated:
Samples displaying feature points are updated to draw additional lip points.
Reduced size of visage|SDK libraries.
Multi-frame analysis feature
VisageFaceAnalyser now offers multi-frame analysis functionality which performs estimations on the selected high-quality frames and returns an averaged, smoothed value. This functionality is available via the VisageFaceAnalyser::analyseStream() function.
New tracking algorithm models
Introducing smaller and faster tracking algorithm models while retaining the same accuracy which completely replace the old models that will no longer be distributed.
New age and gender estimation models
Introducing smaller, faster and more accurate age and gender estimation models which completely replace the old models that will no longer be distributed.
FaceData.getFaceBoundingBox
VisageFaceAnalyser API
VisageFaceAnalyser API was refactored to optimise usage and to include newly introduced multi-frame analysis feature.VisageFaceAnalyser.estimateAge
VisageFaceAnalyser.estimateGender
VisageFaceAnalyser.estimateEmotion
VisageFaceAnalyser.analyseImage(frameWidth, frameHeight, p_imageData, faceData, options, results)
VisageFaceAnalyser.analyseStream(frameWidth, frameHeight, p_imageData, faceData, options, results, faceIndex)
VisageFaceAnalyser.resetStreamAnalysis(faceIndex)
Introducing new AnalysisData structure which contains face analysis results (age, gender and emotions) returned by the VisageFaceAnalyser.analyseImage and VisageFaceAnalyser.analyseStream functions.
VisageDetector API
Prototype of VisageDetector constructor has been changed:VisageDetector()
VisageDetector(configFileName)
VisageGazeTracker API
Prototype of following method has been changed:track(frameWidth, frameHeight, p_imageData, faceDataVector, format, origin, widthStep, timeStamp)
track(frameWidth, frameHeight, p_imageData, faceDataVector, format, origin, widthStep, timeStamp, maxFaces)
Following data files in www/lib folder updated:
ShowcaseDemo and VisageTrackerUnityDemo:
Newly introduced face analysis functions, analyseStream() and resetStreamAnalysis(), are used to perform face analysis, thus replacing previously used estimation functions and in-app filtering.Face tracking and detection algorithms
The default legacy algorithm that could be allowed via the use_vnn configuration parameter (value 0) has been removed.
The VNN algorithm is now the default and only tracking and detection algorithm.
The algorithm can be configured to improve feature points precision and robustness,
over tracking speed (performance) by setting the refine_landmarks parameter to 1 in the configuration file.
Otherwise, when performance is preferred over feature points precision, the recommendation is to disable refine_landmarks parameter by setting it to 0 in the configuration file.
Face tracking and detection with protection masks
Face tracking and detection algorithms are enhanced so that they can track and detect faces wearing protective masks of various colors and patterns.
VisageTrackerUnityPlugin and VisageAnalyserUnityPlugin
Renaming unityInstance to unityGame:
FaceData.getFaceRotationApparent
VisageTrackerUnityPlugin API
Introducing new function for getting the rotation of the face from the camera viewpoint:
VisageTrackerNative._getHeadRotationApparent(rotationApparent, faceIndex)
Removed
Added
Renamed
Modified
Please refer to the VisageTracker Configuration Manual, section 2.1. for additional information.
If you want to update your existing configuration files, it is recommended to copy the parameters values from Facial Features Tracker.cfg configuration file supplied in this package.
Removed
Modified
Added
Projects using older versions of these files should be updated to load the newest data files along with libraries (visageSDK.js, visageSDK.wasm) from the www/lib folder.
Samples displaying feature points are updated to draw physical contour points.
ShowcaseDemo and VisageTrackerUnityDemo:
Newly introduced FaceData class member, faceRotationApparent, is used to check if face rotation is within constraints to successfully run face recognition and face analysis.
Configurable neural network runner
To improve the performance of our algorithms and to support a wider variety of neural network models we are introducing an API and a configurable framework for choosing between different neural network running backends.
As a result, additional configuration file NeuralNet.cfg is now included in visage|SDK (located in www/lib). This file allows the users to configure which backend will be used by visage|SDK. Users can choose between:
New face recognition model
Introducing smaller, faster and more accurate face recognition model.
New face detection model
Introducing a more accurate and robust face detection model. The new model is used in face tracking and face detection when use_vnn configuration parameter is set to 1. Otherwise, the previous face detection model will be used.
For more information about using VNN detection algorithm, please consult VisageTracker Configuration Manual and
VisageDetector documentation.
ASM library format
Depricated ASM library format is no longer supported.
visageAR
visageAR.js library now supports changing screen orientation on mobile devices.
visage|SDK initialization order
The order in which the VisageModule is declared and library and data scripts are included is predefined and should be as follows:
Declare VisageModule
VisageModule has to be declared with all its attributes in charge of preloading of the configuration files, license files and possibly, changing the location of data files, before loading visageSDK.js library.
Include visageSDK.js library
visageSDK.js library has to be included via script tag before any other external data loader script.
Include external data loader scripts
VisageTrackerUnityPlugin API
Introducing changes VisageTrackerUnityPlugin API. The list of added and removed function is as follows:
Added:
VisageTrackerNative._preloadExternalJS(fileURL)
VisageTrackerNative._setDataPath(dataPath)
VisageTrackerNative._preloadAnalysisData(fileURL)
VisageTrackerNative._preloadExternalJS(fileURL)
function
visage|SDK initialization order
Offical samples has been updated according to changes in the visage|SDK initialization order.
VisageTrackerUnityDemo
The sample now requires the path to the visageSDK.js library and external data loader scripts to be specified in Tracker gameObject.use_vnn configuration parameter added to VisageDetector configuration file FaceDetector.cfg, set to value 1.
use_vnn configuration parameter values changed and additional value added:
For detailed description of these changes consult VisageTracker Configuration Manual.
If you want to update your existing tracker configuration files, it is recommended to copy the parameters values from Facial Features Tracker - High.cfg configuration file supplied in this package.
Data files has been saparated to optimize data size.
VisageTracker and VisageDetector in addition to visageSDK.data now require an additional data file depending on the selected algorithm (configuration used).
Projects using older versions of these files should be updated to load the newest data files from the www/lib folder.
VisageConfiguration API
Introducing new API for programatically changing VisageTracker configuration parameters, including class:
used in conjunction with new VisageTracker methods:VisageConfiguration VisageTracker::getConfiguration() and
VisageTracker::setConfiguration(VisageConfiguration configuration)
FDP class:
FDP group 10 (ears) has been extended from 10 to 24 points (12 points per ear) as part of ear tracking feature. Additional changes pertaining ear tracking are documented under paragraphs Model changes, Changes in configuration file and Data files changes.
Please note: FDP files saved with visage|SDK 8.6 will not be backwards compatible with previous versions due to the addition of new FDP points.
VisageTracker class:
VisageTracker::stop()
A new model file has been added for ear tracking functionality - jk_300_wEars. The new model is based on the previous 'jk_300' model and contains an additional 334 polygons and 14 new FDP points in group 10 (10.11 - 10.24).
For detailed description of these changes consult VisageTracker Configuration Manual.
refine_ears parameter added, which is off by default. Toggles the tracking and refinement of ear points (group 10) for Visage Tracker.
Facial Features Tracker - High - With Ears.cfg configuration file introduced specifically for purposes of ear tracking. It is largely identical to the regular high configuration with the exception that it uses 'jk_300_wEars' as the active model, and has the 'refine_ears' parameter set to 1 (on).
Facial Features Tracker - Ultra.cfg configuration file introduced specifically for demonstrating VNN algorithm. It is largely identical to the regular high configuration with the exception that it enables 'use_vnn' option setting the parameter to 1 (on).
For detailed description of these changes consult VisageTracker Configuration Manual.
If you want to update your existing configuration files, it is recommended to copy the parameters values from Facial Features Tracker - High.cfg configuration file supplied in this package.
visageSDK.data file, located in the www/lib folder, has been updated the new model files ('jk_300_wEars.fdp' and 'jk_300_wEars.wfm').
Data files required by VNN algorithm are bundled in new data file visageVNNData.data located in www/lib.
Projects using older versions of these files should be updated to contain the newest data files from the www/lib folder.
VisageTrackerUnityDemo:
VisageTrackerUnityDemo is now distributed only as Unity project. Instructions on how to build and run the provided application can be found here. Additionally, instructions for visage|SDK Unity integration can be found here.smoothing_factors - due to the implementation of new smoothing algorithms in VisageTracker, default values and optimal ranges for this parameter have been changed in all configuration
Please refer to the VisageTracker Configuration Manual, section 2.1. for additional information.
If you want to update your existing configuration files, it is recommended to copy the parameters values from Facial Features Tracker - High.cfg configuration file supplied in this package.
AddType application/wasm .wasm
Asynchronous data loadvisage|SDK library has been modified to be completely asynchronous. As a result certain .data files need to be preloaded by calling FS_createPreloadedFile() function. The change refers to license and configuration files. Note, that the FS_createPreloadedFile() function must be called after visageSDK.js script is loaded, but before the .data is completely downloaded (see Changing the location of the .data file example).
Change the name of the Module Main visage|SDK module renamed from Module to VisageModule in visageAnalysisData.js and visageRecognitionData.js scripts to improve compatilibity with other libararies and software (for example Unity). In case of using FaceAnalysis or FaceRecognition it is necessary to call Module VisageModule and as such use it in the code.
<script src="../../lib/visageSDK.js"></script>
<script>
VisageModule = VisageModule({onRuntimeInitialized: onModuleInitialized});
var preloadFiles = function() {
VisageModule.FS_createPreloadedFile('/', 'Head Tracker.cfg', "../../lib/Head Tracker.cfg", true, false);
VisageModule.FS_createPreloadedFile('/', licenseName, licenseURL, true, false);
};
VisageModule.preRun.push(preloadFiles);
</script>
visageAR
The order of applying rotations on the glasses 3D objects has been fixed to be equal to the tracker's (see getFaceRotation) which is 'YXZ'.
FaceDetector and FaceTracker samples have been removed from visage|SDK HTML5 package.
VisageTrackerUnityPluginVisageTrackerUnityPlugin was updated so it includes all FaceTrack (including multiple face tracking) and FaceAnalysis functions. The plugin is devided into two files: VisageTrackerUnityPlugin.jslib and VisageAnalyserUnityPlugin.jslib, both file can be found in www/lib folder. Due to major changes, it is recommended to go through documentation before including VisageTrackerUnityPlugin to your existing project.
ShowcaseDemo
Improved sample performance by moving FaceAnalysis functionality to a Web Worker (see analysisWorker.js file).Module object renamed to VisageModule. Automatic initialization and data download has been deprecated. In order to properly initialize VisageModule object and start visageSDK data download, VisageModule object needs to be initialized manually.
Following code demonstrates:<script src="../../lib/visageSDK.js"> <script> var Module = VisageModule({onRuntimeInitialized: onModuleInitialized}) </script>See additional examples within sample application code and in the API documentation (API->VisageTracker->Changing the location of the .data file section).
Configuration file format had stayed the same.
jk_300 model has been set as a default model for pose and action unit estimation instead of candide3 model.
If you want to update your existing configuration files, it is recommended to copy the parameters values from Facial Features Tracker - High.cfg configuration file supplied in this package.
VisageTracker:Due to the introduction of multitracking prototype of following methods has been changed:
number VisageTracker.track(number frameWidth, number frameHeight, number p_imageData, FaceData facedata, number format = VISAGE_FRAMEGRABBER_FMT_RGB, number origin = VISAGE_FRAMEGRABBER_ORIGIN_TL, number widthStep = 0, number timeStamp = -1);to:
VectorInt VisageTracker::track(number frameWidth, number frameHeight, number p_imageData, FaceDataVector facedata, number format = VISAGE_FRAMEGRABBER_FMT_RGB, number origin = VISAGE_FRAMEGRABBER_ORIGIN_TL, number widthStep = 0, number timeStamp = -1, number maxFaces = 1);
VisageGazeTracker:Prototype of the VisageGazeTracker' track function has changed and is the same as the VisageTracker' track function.
VisageFaceRecognition:
VisageFaceRecognition.getDescriptorSize();
VisageFaceRecognition.extractDescriptor(FaceData facedata, number frameWidth, number frameHeight, number p_ImageData, VectorShort descriptor); VisageFaceRecognition.descriptorsSimilarity(VectorShort first_descriptor, VectorShort second_descriptor); VisageFaceRecognition.addDescriptor(VectorShort descriptor, string name);
face_detector_sensitivity parameter added. Controls face detector sensitivity (TPR - true positive rate) for VisageFeaturesDetector and for initialization phase of the Visage Tracker.
Pose fitting model sensitivity for z translation(pose_fitting_pose_sensitivity) increased from 0.001 to 0.1
If you want to update your existing configuration files, it is recommended to copy the values for these new parameters from the Facial Features Tracker - High.cfg configuration file supplied in this package. In all other cases, simply use the provided configuration files.
VisageTracker.setTrackerConfigurationFileName(const char* trackerConfigFile)to:
VisageTracker.setTrackerConfigurationFile(const char* trackerConfigFile, bool au_fitting_disabled = false, bool mesh_fitting_disabled = false)
VisageDetector.detectFacialFeatures(VsImage* frame, FaceData* output, int maxFaces = 1, float minFaceScale = 0.1f)to:
VisageDetector.detectFacialFeatures(VsImage* frame, FaceData* output, int maxFaces = 1, float minFaceScale = 0.1f, float maxFaceScale = 1.0f)
process_eyes configuration parameter changed to bit-flag type parameter. Controls gaze vector calculation and pupil points refinement. Setting the parameter to 1 enables the gaze calculations, setting it to 2 enables the pupil refinement and setting it to 3 enables both functionalities.
Configuration parameters min_face_scale and max_face_scale added. If they are not listed in the configuration file default values will be used (0.15,1.0).
Smoothing factor groups have been updated.
New group for face contour smoothing has been added - contour.
Mouth and chin group has been renamed to mouth and now covers only mouth smoothing. Chin smoothing is moved to contour smoothing group.
OLD PARAMETER NAME | POSE FITTING MODEL | AU FITTING MODEL | MESH FITTING MODEL |
model_filename | pose_fitting_model | au_fitting_model | mesh_fitting_model |
fdp_filename | pose_fitting_fdp | au_fitting_fdp | mesh_fitting_fdp |
au_use | pose_fitting_au_use | au_fitting_au_use | mesh_fitting_au_use |
ekf_sensitivity | pose_fitting_au_sensitivity | au_fitting_au_sensitivity | mesh_fitting_au_sensitivity |
- | pose_fitting_su_use | au_fitting_su_use | mesh_fitting_su_use |
- | pose_fitting_su_sensitivity | au_fitting_su_sensitivity | mesh_fitting_su_sensitivity |
If you want to update your existing configuration files, it is recommended to copy the values for these new parameters from the Facial Features Tracker - High.cfg configuration file supplied in this package. In all other cases, simply use the provided configuration files.
Module.getSDKVersion()
DetectStrip API has been removed.
All visage|SDK classes are now part of the global Module object.
initializeLicenseManager() function is no longer a member class of VisageTracker/VisageDetector/VisageFaceAnalyser but a member class of the global Module object:
Module.initializeLicenseManager("license_key_file")
In case of multiple license files call Module.initializeLicenseManager repeatedly.
IMPORTANT: Call this function before the constructor of visage|SDK class in order for licensing to work.Tracker
Tracker class has been renamed to VisageTracker.
this.setConfigurationFile has been renamed to this.setTrackerConfigurationFile.
prototype of the track() function has been changed to:
this.track(frameWidth,frameHeight,p_imageData,faceData,format,origin,widthStep,timeStamp)For additional information on the usage and the extra parameters consult the API documentation.
Detector
Detector class has been renamed to VisageDetector.
prototype of the constructor has been changed:
new Module.VisageDetector();
prototype of the detectFeatures() function has been changed:
this.detectFeatures(frameWidth,frameHeight,p_imageData,faceDataVector,maxFaces,minFaceScale)
FaceAnalyser
FaceAnalyser class has been renamed to VisageFaceAnalyser.
prototype of all the estimate() functions has been changed:
this.estimateAge(frameWidth,frameHeight,p_imageData,faceData); this.estimateGender(frameWidth,frameHeight,p_imageData,faceData); this.estimateEmotion(frameWidth,frameHeight,p_imageData,faceData,prob_estimates);
FaceData
FaceData class has been reimplemented. Functions returning arrays or custom vector containers are prefixed with get:
this.getFaceTranslation(); this.getFaceRotation(); (...) this.getShapeUnits(); (...) this.getFeaturePoints3D();Instead of screenSpaceGazePos member FaceData contains the gazeData member encompassing all gaze relevant information:
ScreenSpaceGazeData.index; ScreenSpaceGazeData.x; ScreenSpaceGazeData.y; ScreenSpaceGazeData.quality; ScreenSpaceGazeData.inState;
IMPORTANT: Check the API documentation for major classes FaceData, FDP, FeaturePoint, ScreenSpaceGazeData usage as well as helper classes such as VectorFloat, VectorInt, VectorString and FaceDataVector. Examples of usage can be found in visage|SDK samples.
Smoothing factors groups and their default values have been changed (translation and rotation groups removed, new groups include eyebrows, mouth_and_chin,
pupils, eyelids and nose).
Some parameters have been removed from configuration files since they are not used anymore.
Please consult VisageTracker Configuration Manual to see complete list of parameters in configuration files.
If you want to update your existing configuration files, it is recommended to copy the values for these new parameters from the Facial Features Tracker - High.cfg configuration file supplied in this package. In all other cases, simply use the provided configuration files.
Due to compiler update to emscripten v1.35, HTML5 samples using visage|SDK must have additional code added to your application main .html file. The additional code must be located inside of the script tag and included before the script tag loading visageSDK.js library. Example:
<script type="text/javascript"> var Module = { onRuntimeInitialized: function() { onModuleInitialized(); } }; </script> <script src="../../lib/visageSDK.js"></script>Emscripten functions that use Module object such as
Module._mallocand visage|SDK.js function such as
m_Tracker.track()must not be called before the OnModuleInitialized() function is called.
Unlicensed VisageDetector will return complete results but has a delay of 2 seconds for each call of detectFacialFeatures method.
Unlicensed VisageFaceAnalyser has a delay of 2 seconds for each call of estimate function.
FFT, HT and Facial Features Tracker - Asymmetric.cfg configuration files have been removed. Two new configurations have been added to the Samples/data folder:
New configuration optional parameters high_prec_addition and tracking_preproc added. If they are not listed in the configuration file default values will be used (1,1).
Smoothing factor groups have been updated. Default values of smoothing factors have been changed in configuration files.
For detailed description of these changes consult VisageTracker Configuration Manual.
All configuration now use candide3.wfm model file. It is recommended to set the model_filename parameter in all existing configurations so it points to candide3.wfm.
If you want to update your existing configuration files, it is recommended to copy the values for these new parameters from the Facial Features Tracker - High.cfg configuration file supplied in this package. In all other cases, simply use the provided configuration files.
An additional parameter has been added to the FaceDetector detectFeatures method. The minFaceSize parameter is used to specify a lower limit to face size (in pixels) in a given image:
this.detectFeatures = function(frameWidth, frameHeight, p_imageData, faceData);changed to:
this.detectFeatures = function(frameWidth, frameHeight, p_imageData, faceData, minFaceSize);
If you want to update your existing configuration files, it is recommended to copy the values for these new parameters from the FFT - HighPerformance.cfg configuration file supplied in this package. In all other cases, simply use the provided configuration files.
Data file is located in the folder where the application html file is located (e.g. www/Samples/FaceTracker folder). Copy the file to appropriate folder in any existing applications.
This section covers all changes introduced in all releases since 7.1.744.
Screen space gaze tracking API introduced.
1. Screen space gaze tracking API methods added to the Tracker class:
this.initOnlineGazeCalibration = function();
this.addGazeCalibrationPoint = function(x, y);
this.finalizeOnlineGazeCalibration = function();
2. Added two members related to screen space gaze tracking to FaceData container class:
this.screenSpaceGazePos = [];
this.screenSpaceGazeState;
If you want to update your existing configuration files, it is recommended to copy the values for these new parameters from the FFT - HighPerformance.cfg configuration file supplied in this package. In all other cases, simply use the provided configuration files.
New sample added: Gaze Tracker, demonstrates online screen space gaze tracking.
Tracking bug on high in plane rotations (z axis) fixed.
this.drawResults = function(p_imageData) method added to the Detector class.
For unlicensed use:
Start and Stop button removed. Tracking will start automatically when tracker is ready and all files have been downloaded.
Firefox freezes when no license is provided bug fixed.
1. FaceData container class for tracking and detection results is introduced. TrackingData class is deprecated.
2. Tracker.track() method is changed to accept FaceData parameter instead of TrackingData parameter. Interface is changed from:
this.track = function(frameWidth,frameHeight,imageData,trackingData)
to:
this.track = function(frameWidth,frameHeight,imageData,faceData)
3. Added 3 methods to the Tracker class:
this.getIPD = function()
this.setIPD = function(IPD)
this.detectStrip = function(IPDPointer)
4. Added Detector class with following methods:
this.detectFeatures = function(frameWidth,frameHeight,imageData,faceData)
this.setConfigurationFile = function(configFilename)
this.initializeLicenseManager = function(licenseKey)