Image Recognition module

Digital asset management Unbundled: Extension

Download Multiple submodules

Edition

DX Core

License

MLA

Issues

IMGREC

Maven site

Image Recognition

Latest

2.0.0

The Image Recognition module searches for untagged images, recognizes and tags them via an external image recognition solution. The module also provides the integration of the image recognition feature with content tags.

Module structure

artifactID Description

magnolia-image-recognition-parent

Parent reactor.

     magnolia-content-tags-integration

Provides functionality to integrate content tags and the image recognition service using decorations.

     magnolia-image-recognition-api

Provides the Java API for image recognition service implementations.

     magnolia-image-recognition

Provides the Magnolia integration, delegating to the chosen service.

     magnolia-amazon-image-recognition

Provides functionality to recognize images via Amazon Rekognition.

Installing with Maven

Maven is the easiest way to install the module. Add the following to your bundle:

<dependency>
  <groupId>info.magnolia.ai.image</groupId>
  <artifactId>magnolia-image-recognition</artifactId>
  <version>2.0.0</version> (1)
</dependency>
1 Should you need to specify the module version, do it using <version>. 
<dependency>
  <groupId>info.magnolia.ai.image</groupId>
  <artifactId>magnolia-content-tags-integration</artifactId>
  <version>2.0.0</version> (1)
</dependency>
1 Should you need to specify the module version, do it using <version>. 
<dependency>
  <groupId>info.magnolia.ai.image</groupId>
  <artifactId>magnolia-image-recognition-api</artifactId>
  <version>2.0.0</version> (1)
</dependency>
1 Should you need to specify the module version, do it using <version>. 
<dependency>
  <groupId>info.magnolia.ai.image</groupId>
  <artifactId>magnolia-amazon-image-recognition</artifactId>
  <version>2.0.0</version> (1)
</dependency>
1 Should you need to specify the module version, do it using <version>.

Amazon Rekognition service configuration

High-level configuration steps:

  • Once the correct permissions are granted, configure the connection to the Amazon Rekognition service.

  • Configure the service as required.

  • Fine-tune the performance of parallel image recognition.

AWS IAM Policy

Make sure that you have acquired appropriate permissions for the service in the AWS IAM Management Console.

AWS IAM Policy

The minimum required permission is read access level and action execution for rekognition:DetectLabels.

Minimum required permission
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "rekognition:DetectLabels" (1)
            ],
            "Resource": "*"
        }
    ]
}
1 Grant access for AWS DetectLabels.

Configuring the AWS connection

The magnolia-aws-foundation module handles all Amazon connections from Magnolia. It’s installed automatically by Maven when you install any AWS-dependent module.

To use AWS in Magnolia, you must have a working AWS account.

You need AWS credentials to connect AWS to Magnolia. Credentials consist of:

  • AWS access key ID

  • AWS secret access key

  • Optionally, a session token (when using the AWS default credential provider chain)

Generate the key in the security credentials section of the Amazon IAM Management Console. In the navigation bar on the upper right, choose your user name, and then choose My Security Credentials. You can store your AWS credentials using:

  • Magnolia Passwords app (session tokens aren’t supported in the app)

  • AWS default credential provider chain

Using the Passwords app

Add your generated access key ID and the secret access key to your Magnolia instance in the Passwords app using the following names and order:

📁 aws-credentials

     aws_access_key_id

     aws_secret_access_key
Using the AWS default credential provider chain

The AWS SDK uses a chain of sources to look for credentials in a specific order. For more information, see Default credentials provider chain.

  1. Set your AWS credentials by following the instructions in the AWS documentation: Provide temporary credentials to the SDK.

    For a more secure implementation using the default credential provider chain, we recommend using a session token, which expires, rather than a permanent user token.

  2. Disable Magnolia’s internal credential handling by doing one of the following:

    1. Adding the following configuration properties to your WEB-INF/config/default/magnolia.properties file:

      magnolia.aws.validateCredentials=false
magnolia.aws.useCredentials=false

    2. Using JVM arguments as shown in the next step.

  3. Set your AWS session or user token. AWS credentials can be injected using environment variables or JVM system properties. For more details, see Default credentials provider chain and Configure access to temporary credentials.

    Example configuration with a session token and JVM arguments
    -Dmagnolia.aws.validateCredentials=false(1)
-Dmagnolia.aws.useCredentials=false(1)
-Daws.accessKeyId=$AWS_ACCESS_KEY_ID(2)
-Daws.secretAccessKey=$AWS_SECRET_ACCESS_KEY(2)
-Daws.sessionToken=$AWS_SESSION_TOKEN(2)(3)
    1 Disables Magnolia’s internal credential handling using JVM properties.
    2 JVM properties to inject environment variables containing the AWS credentials. Ensure that your environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN are set.
    3 AWS_SESSION_TOKEN is optional.
    Example configuration with a permanent user token
    -Dmagnolia.aws.validateCredentials=false
-Dmagnolia.aws.useCredentials=false
-Daws.accessKeyId=<your-access-key-id>
-Daws.secretAccessKey=<your-secret-access-key>

Set the service provider

Set the service provider in /image-recognition/config.yaml.

/image-recognition/config.yaml
currentService: info.magnolia.ai.image.amazon.AmazonImageRecogniser
Property Description

currentService

required, default is `info.magnolia.ai.image.amazon.AmazonImageRecogniser`

The class name of the service to be used.

Options:

  • info.magnolia.ai.image.amazon.AmazonImageRecogniser

Region name

You need to know a region name to configure the Amazon Rekognition Image service in Magnolia. To reduce data latency, AWS offers several regional endpoints. Each of the endpoints can be referred to in service configurations by a region name, for example eu-west-1.

For more information, see Amazon’s AWS Regions and Endpoints page.

If you pick a region that doesn’t support this service, you may get erratic results.

Configuration options

Under /amazon-image-recognition/config.yaml, you must configure the following properties for the recognition service:

region:
  name: your_aws_region_name
maxLabels: 10
minConfidence: 50
supportedFormats:
  png: png
  jpg: jpg
  jpeg: jpeg

Properties

Property Description

region name

required

Label designating a regional endpoint to which the image recognition service connects, such as eu-west-1.

For a list of available regions and labels, see AWS: Regional endpoints.

maxLabels

required, default is 10

The maximum number of tags you want the recognition service to return.

This is an integer and the minimum value is 0.

If 0 is used, no tag would be assigned to an image asset.

minConfidence

required, default is `50`

The confidence score of recognition.

The Amazon Rekognition service returns a confidence score for each image tag. Image tags with a recognition confidence lower than the value of the minConfidence property are dropped.

supportedFormats

required, default are png, jpg, jpeg

A list of image formats defining which image types are automatically recognized by the image recognition service.

The Amazon Rekognition service currently supports only two image formats: JPEG/JPG and PNG.

Be aware that the Amazon Rekognition service has several service limits which may influence the performance of the recognition process. For more information, see Guidelines and quotas in Amazon Rekognition.

Parallel recognition threads

Image recognition can be an intensive process in terms of CPU resources. The recognition process can be delegated to several parallel threads. With more threads working you can obtain the results faster.

You can specify the number of parallel image recognition threads for the image recognition service provided by the image-recognition-api submodule. Set the number in the magnolia.properties configuration file through a Magnolia property called magnolia.image.recognition.numberOfThreads.

Example
magnolia.image.recognition.numberOfThreads=3 (1)
1 Sets three recognition threads. If the property isn’t set in the file, the default number of threads used by the system is 1.

Using other image recognition solutions

You can implement another image recognition solution or integrate your app with another third-party image recognition solution using the info.magnolia.ai.image.ImageRecogniser interface provided by the image-recognition-api submodule:

    /**
     * Takes image bytes as parameter and returns a {@link Collection collection}
     * of {@link ImageLabel Image label}s as output.
     *
     * <p>
     * Returns empty collection for the cases below:
     * <li>Upon exception</li>
     * <li>Image couldn't be recognized</li>
     * </p>
     */
    Collection<ImageLabel> recognise(byte[] imageBytes);

Recognizing images

The image recognition and tagging action is executed during the startup of the author instance and every time you upload a new image asset.

You can also trigger the action manually in the Assets app by selecting one or more assets and clicking the Run recognition action.

Images that have already been tagged are marked as such using a JCR property called lastTaggingAttemptDate. Executing the manual Run recognition action forces a new tag to be set even if the image was previously tagged.

The image recognition feature is available only on author instances.

Removing tags

Once an image has been tagged, you can remove some or all of the tags by selecting the asset and clicking the Modify tags action in the Assets app.

In the dialog box that opens, you can remove individual tags or click Remove all tags.

Related topics
Feedback

DX Core

×

Location

This widget lets you know where you are on the docs site.

You are currently perusing through the DX Core docs.

Main doc sections

DX Core Headless PaaS Legacy Cloud Incubator modules