Class: VisageFaceRecognition

VisageFaceRecognition

new VisageFaceRecognition()

VisageFaceRecognition class contains a face recognition algorithm capable of measuring similarity between human faces and recognizing a person's identity from frontal facial images (yaw angle approximately from -20 to 20 degrees) by comparing face descriptors.

The face descriptor is a condensed representation of a face, stored in a VectorShort object. The size of descriptor can be obtained by calling function getDescriptorSize(), from now on in the text referred to as size_descriptor.

To extract descriptors from facial image, method extractDescriptor() is provided. Similarity between two faces is calculated as distance between their face descriptors and returned as a value between 0 (no similarity) and 1 (maximum similarity) using descriptorsSimilarity().

For a person to be recognized, its identity has to be stored beforehand. Database of stored face descriptors will be referred to as face recognition gallery.
For smaller sets of face descriptors (up to several hundred), higher-level API for recognizing person's identity using face recognition gallery is provided. The gallery is an array of face descriptors, with a corresponding array of names, so that the gallery may contain n descriptors and a corresponding name for each descriptor. To perform recognition, the face descriptor extracted from the input image is then compared with face descriptors from the gallery and the similarity between the input face descriptor and all face descriptors from the gallery is calculated in order to find the face(s) from the gallery that are most similar to the input face. The VisageFaceRecognition class includes the following set of functions for manipulating the gallery, including adding, removing, naming descriptors in the gallery as well as loading and saving the gallery to file:
The gallery is stored in IndexedDB browser's database system. This storage is persistent across sessions until it is explicitly deleted within the browser. The gallery is saved as a simple txt file under /vis_data folder and as such it is not intended to be used to save large number of face descriptors. The current implementation of gallery is only a demonstration of how Face Recognition API can be used and it is recommended to implement more efficient solution for scenarios which require large face recognition gallery.



Face recognition is based on the FaceData obtained from the VisageTracker or VisageDetector API.

Note: After the end of use VisageFaceRecognition object needs to be deleted to release the allocated memory. Example: 

<script>
m_Recognition = new VisageModule.VisageFaceRecognition();
...
m_Recognition.delete();
</script>


Dependencies

VisageFaceRecognition requires algorithm data file, neural network configuration file and license key file to be preloaded to virtual file system. Data and neural network configuration file can be found in the www/lib folder.

Data files
An external loader script visageRecognitionData.js is provided for preloading the visageRecognitionData.data file.

Changing the location of data files
By default, loader scripts expect the .data files to be in the same location as the application's main html file, while visageSDK.wasm is expected to be in the same location as visageSDK.js library file. However, location of the .data and .wasm files can be changed.
The code example below shows how to implement locateFile function and how to set it as an attribute to the VisageModule object.

Configuration file and license key files
Configuration file and the license key files are preloaded using VisageModule's API function assigned to the preRun attribute: 

      VisageModule.FS_createPreloadedFile(parent, name, url, canRead, canWrite)
where parent and name are the path on the virtual file system and the name of the file, respectively.



visage|SDK initialization order

The order in which the VisageModule is declared and library and data scripts are included is important.
  • First, VisageModule object is declared
    • including preloading of the configuration files, license files and possibly, changing the location of data files
  • then visageSDK.js library script is included and
  • last, visageRecognitionData.js external data loader script is included


Sample usage - changing data files location and script including order: 

<script>
licenseName = "lic_web.vlc"
licenseURL = "lic_web.vlc"

var locateFile = function(dataFileName) {var relativePath = "../../lib/" + dataFileName; return relativePath};

VisageModule = {

  locateFile: locateFile,

  preRun: [function() {
      VisageModule.FS_createPreloadedFile('/', 'NeuralNet.cfg', "../../lib/NeuralNet.cfg", true, false);
      VisageModule.FS_createPreloadedFile('/', 'Head Tracker.cfg', "../../lib/Head Tracker.cfg", true, false);
      VisageModule.FS_createPreloadedFile('/', licenseName, licenseURL, true, false, function(){ },  function(){ alert("Loading License Failed!") }); 	

  }],

  onRuntimeInitialized: onModuleInitialized
}
</script>


<script src="../../lib/visageSDK.js"> </script>
<script src="../../lib/visageRecognitionData.js"> </script>



VisageFaceRecognition class implementation in Web Worker

VisageFaceRecognition class can be implemented in a separate thread in relation to the VisageTracker class resulting in better performance.

To send the whole FaceData object from VisageTracker to Recognition Worker, convert FaceData object use one of the provided conversion pair functions:

Note: The worker thread as well as the main thread has to load the visageSDK.js script and additional visageRecognitionData.js.

Methods

extractDescriptor(faceData, frameWidth, frameHeight, p_imageData, descriptor) → {number}

Extracts the face descriptor for face recognition from a facial image. Prior to using this function, it is neccessary to process the facial image or video frame using VisageTracker or VisageFeaturesDetector and pass the obtained facial data to this function.


Sample usage: 


// ... initialize visageFaceRecognition object ...
// ... create a face data object ...
// ... initialize and call VisageTracker/VisageFeaturesDetector on the image ...
//
// construct a VectorShort object to pass to the extractDescriptor method
var descriptor = new Module.VectorShort();
//
// call extractDescriptor and pass the faceData input parameter, image data and information and a short vector to be populated
var success = visageFaceRecognition.extractDescriptor(faceData, mWidth, mHeight, ppixels, descriptor);
if (success)
{
  // ... descriptor is populated and ready to be used ...
  // read the first element
  var first = descriptor.get(0);
}
// when done using descriptor clear memory
descriptor.delete();


Parameters:
Name Type Description
faceData FaceData FaceData instance. Facial data obtained from VisageTracker or VisageFeaturesDetector.
frameWidth number Width of the frame
frameHeight number Height of the frame
p_imageData number Pointer to image pixel data, size of the array must correspond to frameWidth and frameHeight
descriptor VectorShort size_descriptor-dimensional array of short. The resulting face descriptor is returned in this array.
See:
Returns:
status - 1 on success, 0 on failure.

Type
number

descriptorsSimilarity(first_descriptor, second_descriptor) → {number}

Calculates similarity between two descriptors.

The function returns a float value between 0 and 1. Two descriptors are equal if the similarity is 1. Two descriptors are completely different if the similarity is 0.

Parameters:
Name Type Description
first_descriptor VectorShort VectorShort object, size of size_descriptor, obtained from extractDescriptor function which contains the first face descriptor information.
second_descriptor VectorShort VectorShort object, size of size_descriptor, obtained from extractDescriptor function which contains the second face descriptor information
See:
Returns:
similarity - Value between 0 and 1, 1 means full similarity and 0 means full diversity.

Type
number

recognize(descriptor, n, names, similarities) → {number}

Compare a face to all faces in the current gallery and return n names of the most similar faces.

Parameters:
Name Type Description
descriptor VectorShort VectorShort object, size of size_descriptor, obtained from extractDescriptor function which contains the first face descriptor information.
n number Number of names and similarities to be returned.
names VectorString VectorString object containing names of n faces from the gallery that are most similar to the input image, ascending by similarity.
similarities VectorFloat VectorFloat object. The function will return the similarity values for the n most similar faces in this array, corresponding to the names array. The values are sorted, with the largest similarity value first.
See:
Returns:
compared_faces - number of compared faces.

Type
number

addDescriptor(faceData, frameWidth, frameHeight, p_imageData, name) → {number}

Extracts a face descriptor from the input RGBA image and adds it to the gallery.

Parameters:
Name Type Description
faceData FaceData FaceData instance. Input parameter.
frameWidth number Width of the frame
frameHeight number Height of the frame
p_imageData number Pointer to image pixel data, size of the array must correspond to frameWidth and frameHeight
name string Descriptor name to be added to the gallery.
Returns:
success - 1 on success, 0 on failure. The function may fail if the face is not found in the image.

Type
number

addDescriptor(descriptor, name) → {number}

Adds face descriptor to the gallery.

Parameters:
Name Type Description
descriptor VectorShort VectorShort object, size of size_descriptor, obtained from extractDescriptor function which contains the first face descriptor information.
name string Descriptor name to be added to the gallery.
Returns:
success - 1 on success, 0 on failure.

Type
number

getDescriptorCount() → {number}

Returns number of descriptors in the gallery.

Returns:
num_descriptors - Number of descriptors in the gallery

Type
number

getDescriptorName(index) → {string}

Returns the name of a descriptor at the given index in the gallery.

Parameters:
Name Type Description
index number Gallery index
Returns:
name - Name of the descriptor at the given index in the gallery.

Type
string

getDescriptorSize() → {number}

Returns size of an array which should be allocated for storing the descriptor.

Returns:
size_descriptor - Size of descriptor

Type
number

replaceDescriptorName(name, index) → {number}

Replaces the name of a descriptor at the given index in the gallery with new name.

Parameters:
Name Type Description
name string New descriptor name
index number Index of descriptor in the gallery
Returns:
success - 1 on success, 0 on failure. The function may fail if index is out of range.

Type
number

removeDescriptor(index) → {number}

Removes a descriptor at the given index from the gallery. The remaining descriptors in the gallery above the given index (if any) are shifted down in the gallery array by one place, filling the gap created by removing the descriptor.

Parameters:
Name Type Description
index number Index of descriptor in the gallery
Returns:
success - 1 on success, 0 on failure. The function may fail if index is out of range.

Type
number

saveGallery(file_name, callback)

Save gallery to the file.

The file format:
<name_of_the_face_in_the_image_1>;<elements_of_the_getDescriptorSize()_dimensional_face_descriptor_separated_by_a_space>
<name_of_the_face_in_the_image_2>;<elements_of_the_getDescriptorSize()_dimensional_face_descriptor_separated_by_a_space>
(...)
<name_of_the_face_in_the_image_n>;<elements_of_the_getDescriptorSize()_dimensional_face_descriptor_separated_by_a_space>


This function is asynchronous and uses Emscripten IDBFS file system API. IDBFS uses browsers' IndexedDB in order to persist data.

Since saveGallery is asynchronous a callback function needs to be provided to notify the user when the function is complete. The callback function is expected to have two parameters - name and status. Parameter name returns name of txt file in which gallery is saved while parameter status returns 1 on success and 0 on function failure.
Sample usage: 

visageFaceRecognition.saveGallery("test_recognition.txt", 
function(name, status)
{   
   // ... do something based on status ...
}
);


Parameters:
Name Type Description
file_name string Name of the file (including path if needed) from which the gallery will be loaded.
callback function Callback function which will be called when saving is complete

loadGallery(file_name, callback)

Load gallery from file. The entries from the loaded gallery are appended to the current gallery. If it is required to replace existing gallery with the loaded one, call resetGallery() first.

This function is asynchronous and uses Emscripten IDBFS file system API. IDBFS uses browsers' IndexedDB in order to persist data.

Since loadGallery is asynchronous a callback function needs to be provided to notify the user when the function is complete. The callback function is expected to have two parameters - name and status. Parameter name returns name of txt file in which gallery is saved while parameter status returns 1 on success and 0 on function failure.

Sample usage: 

visageFaceRecognition.loadGallery("test_recognition.txt", 
function(name, status)
{   
   // ... do something based on status ...
}
);


Parameters:
Name Type Description
file_name string Name of the file (including path if needed) from which the gallery will be loaded.
callback function Callback function which will be called when loading is complete

resetGallery()

Clear all face descriptors from the gallery.