This is a NetX Labs feature! We couldn’t wait to get your feedback on it, so we are providing you early access, even though there is still room for improvement. Don't hesitate to let us know your thoughts about this feature in the NetX Ideas Portal.
Overview
Integrating NetX with Google Vision allows you to auto-tag image assets based on content that is detected by the Google Vision engine. The following Google Vision functions are supported:
- Label Detection – Identifies broad categories within an image, such as modes of transportation or animals. The confidence score threshold is 75% (tags with a confidence score of less than 75% are ignored and will not be tagged in NetX).
- Logo Detection – Identifies common product logos within your images.
- Landmark Detection – Identifies popular natural and man-made landmarks.
- Face Detection – Detects multiple faces in a single image, as well as facial attributes which may indicate emotional state as well as identifying headwear. Facial recognition is not supported.
-
Extract Text — Optical Character Recognition (OCR) enables you to detect text within your images.
For more information, please refer to Google's official documentation.
Setup
Properties
Please note: Correct configuration of the following property is mandatory to use this feature. However, if your instance is hosted by NetX rather than on-premise please get in touch with support before configuring this property.
| Month | Savings |
|---|---|
external.google.vision.credentialFilePath |
This will be the full file path for the Google Service Credential JSON file. To obtain this file, see the section on creating your Google Service account. Value options: JSON file path Requires restart? Yes |
Google Service Account
To use Google Vision integration, you must first create and use a Google account and obtain a service account key:
- Follow along with Google's official setup instructions. You will only need steps 1-3 under "Set up your project"; you will not need a Cloud Storage bucket.
- Next, follow along with Google's steps to create a Service Account JSON key. Begin with "Set up a service account". You are ready to proceed once you have a JSON Service Account key file.
If you have an on-premise deployment, you'll have to install the JSON credential file on the NetX server and then configure the Google Vision property to reflect the path to that file. If you are using NetX SaaS, then you'll want to work with Support to securely transfer your JSON credential file to NetX, and Support will install and configure this file for you.
Implementation
AutoTask
To implement Google Vision with your NetX assets, you must use NetX's AutoTask system. You may customize your AutoTask by applying filters that may limit those assets that are auto-tagged, such as by specifying a category or folder, or even configure multiple AutoTask functions with different parameters.
The example shown below is one implementation that will tag all image assets (e.g. "fileFormatFamily" with a value of "image"). This will apply to those images being imported and will tag those imported assets within the specified attribute field ("Labels"). This particular AutoTask specifies that Google Vision will look for and tag landmarks or labels up to 10 new attributes, separated by a semicolon:
<matchCriteria type="and">
<criteria type="action" value="import"></criteria>
<criteria name="fileFormatFamily" type="attribute" value="image"></criteria>
</matchCriteria>
<customJob attrName="Labels" className="com.netxposure.products.imageportal.autotask2.impl.GoogleVisionExtractImageJob" delimiter="semi" faces="false" labels="true" landmarks="true" logos="false" text="false" maxLabels="10"></customJob>For Google Vision and its configured AutoTask to take effect, restarting NetX is required. Additionally, if you wish to retroactively apply Google Vision tags to existing attributes, resyncing or reimporting may be required, depending on your AutoTask.
AutoTask Parameters
The example above can be customized using the following parameters:
If an Autotask does not include a faces, labels, landmarks, logos, or text parameter set to "true", no tagging will take place.
| Name | Values | Description |
|---|---|---|
faces |
true / false | Whether or not facial expressions will be part of Google Vision's analysis. |
labels |
true / false | Whether or not labels will be part of Google Vision's analysis. |
landmarks |
true / false | Whether or not landmarks will be part of Google Vision's analysis. |
logos |
true / false | Whether or not logos will be part of Google Vision's analysis. |
text |
true / false | Whether text from your images will be pulled into your attribute data. |
maxlabels |
Number | Maximum number of labels Google will return. This maximum applies to each Google Vision function (not the aggregate total). |
attrName |
Text or text area attribute | Name of a pre-existing Attribute; its type should be text or textArea. |
delimiter |
comma / space / semi | This will determine the delimeter used for your list of labels; accepted values are comma, semi (for semicolon), or space. |
Proxies
For your photos to be properly tagged using Google Vision, the asset in question must have a proxy file. This may be, in order of preference, a Zoom, Preview, or Thumbnail. This is due to proxy files being more efficient as well as prevent any formatting issues (all proxy files are formatted as JPGs).
In Action
Using the AutoTask example above, navigate to the AutoTask tab of the Systems area, create a new task, and paste the content in between the subsequent <task></task> brackets:
Once you have saved your AutoTask, your instance will require a restart for the AutoTask to take effect. After a restart has been performed, you may begin importing new image assets and see the Google Vision attributes being applied to the Attribute field indicated by your AutoTask. However, your Attribute values may not be immediately available in your selected Attribute, and may take up to a couple of minutes to see the values reflected.