Class: VisageDetector

VisageDetector

new VisageDetector(configurationName)

Faces and facial features detector implementation.

VisageDetector class detects one or more faces and their facial features in an image. The input is image data and image descriptors (image width, image height, ...). The results are, for each detected face, the 3D head pose, the coordinates of facial feature points, e.g. chin tip, nose tip, lip corners etc. and 3D face model fitted to the face. The results are returned in one or more FaceData objects, one for each detected face. Please refer to the FaceData documentation for detailed description of returned data.

Note: After the end of use VisageDetector object needs to be deleted to release the allocated memory. Example:

<script>
m_Detector = new VisageModule.VisageDetector("../../lib/Face Detector.cfg");
...
m_Detector.delete();
</script>


Dependencies

VisageDetector requires data file, configuration file and license key file to be preloaded to virtual file system. Data and configuration files can be found in the www/lib folder.

Data files
Main VisageDetector data is bundled in the visageSDK.data file and loaded using the visageSDK.js script.

Changing the location of data files
By default, loader scripts expect the .data files to be in the same location as the application's main html file, while visageSDK.wasm is expected to be in the same location as visageSDK.js library file. However, location of the .data and .wasm files can be changed.
The code example below shows how to implement locateFile function and how to set it as an attribute to the VisageModule object.

Configuration and license key files
VisageDetector uses Face Detector.cfg configuration file. Configuration and the license key files are preloaded to the virtual file system using VisageModule's API function assigned to the preRun attribute:

      VisageModule.FS_createPreloadedFile(parent, name, url, canRead, canWrite)
where parent and name are the path on the virtual file system and the name of the file, respectively.

visage|SDK initialization order

The order in which the VisageModule is declared and library and data scripts are included is important.
  • First, VisageModule object is declared
    • including preloading of the configuration files, license files and possibly, changing the location of data files
  • then visageSDK.js library script is included and
  • last, external data loader script is included


Sample usage - changing data files location and script including order:

<script>
licenseName = "lic_web.vlc"
licenseURL = "lic_web.vlc"

var locateFile = function(dataFileName) {var relativePath = "../../lib/" + dataFileName; return relativePath};

VisageModule = {

  locateFile: locateFile,

  preRun: [function() {
      VisageModule.FS_createPreloadedFile('/', 'Face Detector.cfg', "../../lib/Face Detector.cfg", true, false);
      VisageModule.FS_createPreloadedFile('/', 'NeuralNet.cfg', "../../lib/NeuralNet.cfg", true, false);
      VisageModule.FS_createPreloadedFile('/', licenseName, licenseURL, true, false, function(){ },  function(){ alert("Loading License Failed!") }); 	

  }],

  onRuntimeInitialized: onModuleInitialized
}
</script>

<script src="../../lib/visageSDK.js"> </script>




Parameters:
Name Type Description
configurationName string the name of the detector configuration file (Face Detector.cfg; default configuration file provided in lib folder; for further details see VTCM).

Methods

detectFeatures(frameWidth, frameHeight, p_imageData, faceDataArray, maxFaces, minFaceScale, maxFaceScale, outputOnly2DFeatures) → {number}

Performs faces and facial features detection in a still image.

The algorithm detects one or more faces and their features. The results are, for each detected face, the 3D head pose, gaze direction, eye closure, coordinates of facial feature points (e.g. chin tip, nose tip, lip corners etc.) and 3D face model fitted to the face.

The results are returned in form of FaceData objects. An array of FaceData objects passed to this method as output parameter should be allocated to maxFaces size.

Sample usage:

var m_Detector,
    faceData,
    maxFaces,
    frameWidth,
    frameHeight;

function onInitializeDetector(){
    //Initialize licensing with the obtained license key file
    VisageModule.initializeLicenseManager("xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx.vlc");
    //Instantiate detector object
    m_Detector = new VisageModule.VisageDetector("../../lib/Face Detector.cfg");
    //Instantiate an FaceDataVector instance where detection results will be stored
    faceDataArray = new VisageModule.FaceDataVector();
    //Specify the maximum number of faces that will be detected in the image
    maxFaces = 10;
    for (var i = 0; i < maxFaces; ++i)
    {
          faceData = new VisageModule.FaceData();
          faceDataArray.push_back(faceData);
    }
    frameWidth = canvas.width;
    frameHeight = canvas.height;
    
    //Allocate memory for image data
    ppixels = VisageModule._malloc(mWidth*mHeight*4);
    //Create a view to the memory
    pixels = new Uint8ClampedArray(VisageModule.HEAPU8.buffer, ppixels, mWidth*mHeight*4);
}
function onDetectInImage(){
    //Obtain the image pixel data
    var imageData = canvas.getContext('2d').getImageData(0,0, mWidth, mHeight).data;
    //Fill pixels with imageData
    pixels.set(imageData);
    //Call the detection method
    var numberOfFaces = m_Detector.detectFeatures(frameWidth, frameHeight, ppixels, faceDataArray, maxFaces);
    //Based on the number of detected faces do some action with the return values located in face data array
    if (numberOfFaces > 0)
    {
         for (var i = 0; i < maxFaces; ++i)
         {
              drawSomething(faceDataArray.get(i));
         }
    }    
}
function onCleanUp(){
    //Clean up Detector memory
    m_Detector.delete();
    //Clean up FaceData objects
    for (var i = 0; i < maxFaces; ++i)
    {
          var f_data = faceDataArray.get(i);
          f_data.delete();
    }
    //Clean up FaceDataArray object
    faceDataArray.delete();
}


After this call, n contains the number of faces actually detected. The first n members of the data array are filled with resulting data for each detected face. Please refer to the FaceData documentation for detailed description of returned parameters. If maxFaces is smaller than the number of faces actually present in the image, the function will return only first maxFaces detected faces.

Following image formats are supported:
- VisageModule.VISAGE_FRAMEGRABBER_FMT_RGB: each pixel of the image is represented by three bytes representing red, green and blue channels, respectively.
- VisageModule.VISAGE_FRAMEGRABBER_FMT_RGBA: each pixel of the image is represented by four bytes representing red, green, blue and alpha (ignored) channels, respectively.
- VisageModule.VISAGE_FRAMEGRABBER_FMT_LUMINANCE: each pixel of the image is represented by one byte representing the luminance (gray level) of the image. Origin must be:
- VisageModule.VISAGE_FRAMEGRABBER_ORIGIN_TL: Origin is the top left pixel of the image. Pixels are ordered row-by-row starting from top left.

Note that the input image is internally converted to grayscale.
Parameters:
Name Type Argument Default Description
frameWidth number Width of the frame.
frameHeight number Height of the frame.
p_imageData number Pointer to image pixel data, size of the array must correspond to frameWidth and frameHeight.
faceDataArray FaceDataVector Array of FaceData objects that will receive the detection results. The size of the faceDataArray is equal to maxFaces parameter.
maxFaces number <optional>
1 Maximum number of faces to be detected.
minFaceScale number <optional>
0.1 Scale of smallest face to be searched for, defined as decimal fraction [0.0 - 1.0] of input image size (min(width, height)).
maxFaceScale number <optional>
1.0 Scale of largest face to be searched for, defined as decimal fraction [0.0 - 1.0] of input image size (min(width, height))
outputOnly2DFeatures boolean <optional>
false If set, detection time will be reduced and only featurePoints2D will be returned.
See:
Returns:
numberOfFaces - Number of detected faces (0 or more), -1 if error occurred
Type
number

detectFaces(frameWidth, frameHeight, p_imageData, VSRectVector, maxFaces, minFaceScale, maxFaceScale, useRefinementStep) → {number}

Performs face detection in a still image.

The algorithm detects one or more faces. For each detected face a square facial bounding box is returned.

The results are returned in form of VSRect objects. An array of VSRect objects passed to this method as output parameter should be allocated to maxFaces size.

Sample usage:

var m_Detector,
    faceData,
    maxFaces,
    frameWidth,
    frameHeight;

function onInitializeDetector(){
    //Initialize licensing with the obtained license key file
    VisageModule.initializeLicenseManager("xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx.vlc");
    //Instantiate detector object
    m_Detector = new VisageModule.VisageDetector("../../lib/Face Detector.cfg");
    //Instantiate an VSRectVector instance where detection results will be stored
    boundingBoxArray = new VisageModule.VSRectVector();
    //Specify the maximum number of faces that will be detected in the image
    maxFaces = 10;
    for (var i = 0; i < maxFaces; ++i)
    {
          boundingBox = new VisageModule.VSRect();
          boundingBoxArray.push_back(boundingBox);
    }
    frameWidth = canvas.width;
    frameHeight = canvas.height;
    
    //Allocate memory for image data
    ppixels = VisageModule._malloc(mWidth*mHeight*4);
    //Create a view to the memory
    pixels = new Uint8ClampedArray(VisageModule.HEAPU8.buffer, ppixels, mWidth*mHeight*4);
}
function onDetectInImage(){
    //Obtain the image pixel data
    var imageData = canvas.getContext('2d').getImageData(0,0, mWidth, mHeight).data;
    //Fill pixels with imageData
    pixels.set(imageData);
    //Call the detection method
    var numberOfFaces = m_Detector.detectFaces(frameWidth, frameHeight, ppixels, boundingBoxArray, maxFaces);
    //Based on the number of detected faces do some action with the return values located in VSRect array
    if (numberOfFaces > 0)
    {
         for (var i = 0; i < maxFaces; ++i)
         {
              drawSomething(boundingBoxArray.get(i));
         }
    }    
}
function onCleanUp(){
    //Clean up Detector memory
    m_Detector.delete();
    //Clean up VSRect objects
    for (var i = 0; i < maxFaces; ++i)
    {
          var boundingBox = boundingBoxArray.get(i);
          boundingBox.delete();
    }
    //Clean up VSRectVector object
    boundingBoxArray.delete();
}


After this call, n contains the number of detected faces. The first n members of the faces array are filled with resulting bounding boxes for each detected face. If maxFaces is smaller than the number of faces actually detected in the image, the function will return only first maxFaces detected faces.

Following image formats are supported:
- VisageModule.VISAGE_FRAMEGRABBER_FMT_RGB: each pixel of the image is represented by three bytes representing red, green and blue channels, respectively.
- VisageModule.VISAGE_FRAMEGRABBER_FMT_RGBA: each pixel of the image is represented by four bytes representing red, green, blue and alpha (ignored) channels, respectively.
- VisageModule.VISAGE_FRAMEGRABBER_FMT_LUMINANCE: each pixel of the image is represented by one byte representing the luminance (gray level) of the image. Origin must be:
- VisageModule.VISAGE_FRAMEGRABBER_ORIGIN_TL: Origin is the top left pixel of the image. Pixels are ordered row-by-row starting from top left.

Note that the input image is internally converted to grayscale.
Parameters:
Name Type Argument Default Description
frameWidth number Width of the frame
frameHeight number Height of the frame
p_imageData number Pointer to image pixel data, size of the array must correspond to frameWidth and frameHeight
VSRectVector VSRectVector Provide an empty array. It will be filled with VSRect instances.
maxFaces number <optional>
1 Maximum number of faces to be detected
minFaceScale number <optional>
0.1 Scale of smallest face to be searched for, defined as decimal fraction [0.0 - 1.0] of input image size (min(width, height)).
maxFaceScale number <optional>
1.0 Scale of largest face to be searched for, defined as decimal fraction [0.0 - 1.0] of input image size (min(width, height))
useRefinementStep boolean <optional>
true If set to true, additional refinement algorithm will be used resulting with more precise facial bounding boxes and lower FPR, but higher detection time.
See:
Returns:
numberOfFaces - Number of detected faces (0 or more), -1 if error occurred
Type
number