Use the following information to configure Search Enterprise.

The Admin console is used to manage the interaction between Alfresco and Search Enterprise from the Alfresco Repository. This gives you the ability to determine the high-level health of the Search Enterprise index.

To use Search Enterprise with the Alfresco Content Services platform the following configuration must be applied:

• For searching features the Alfresco Repository properties must be configured in the alfresco-global.properties file. This can also be done as an environment variable by configuring the Search Subsystem.
• The Elasticsearch connector environment variables related to communication with the Alfresco Repository (Database, ActiveMQ and Transform Service) must be set and the Elasticsearch server for indexing features.

Note: To ensure backward compatibility, the exact same property values are used for configuring connection to the Opensearch Search subsystem (‘elasticsearch’ prefixes and aliases shall not change).

Alfresco Repository

Alfresco Repository provides configuration properties for the Elasticsearch Search subsystem that defines the connection to the external Elasticsearch server, for more see Subsystem.

Additional property values can be included in the global configuration file alfresco-global.properties

Some of the properties above can be edited in the Search Admin Console, but values will be applied only to the Alfresco Repository instance. To update values for the Elasticsearch connector update its property file manually. Note: It is important that the Elasticsearch connector and repository configuration match, otherwise the search functionality will be impaired.

Additionally, these properties can be set as environment variables in Alfresco Repository Docker Image when using Docker Compose. In the following sample, elasticsearch.host and elasticsearch.createIndexIfNotExists override the default values.

Code copied to clipboard.
alfresco:
    image: quay.io/alfresco/alfresco-content-repository:23.1.0
    environment:
        JAVA_OPTS: "
        -Dindex.subsystem.name=elasticsearch
        -Delasticsearch.host=elasticsearch
        -Delasticsearch.createIndexIfNotExists=true
        "

Alfresco Elasticsearch connector

The indexing feature is provided by a Spring Boot application called Elasticsearch connector. This application includes two main components that build and maintain the index in Elasticsearch:

• Re-Indexing: Indexing the information of a pre-populated Alfresco Repository or catching up with Alfresco Repositories that has missed some ActiveMQ messages is provided by the re-indexing component.

• Live Indexing: Metadata, and Content and Permissions from the Alfresco Repository are consumed using ActiveMQ messages so they can be indexed in the Elasticsearch server.

Alfresco Re-indexing app

Alfresco re-indexing app requires a working Alfresco Repository Database and the Elasticsearch server.

The tool may be used as a standalone jar file. The table below lists the main configuration properties that can be specified through the Spring Boot configuration.

There are two strategies to fill the gaps in the Elasticsearch server when provoked by ActiveMQ unavailability or any other external cause:

• Fetch by IDS alfresco.reindex.jobName=reindexByIds: index nodes in an interval of database ALF_NODE.id column
• Fetch by DATE alfresco.reindex.jobName=reindexByDate: index nodes in an interval of database ALF_TRANSACTION.commit_time_ms column

Sample invocation for Fetch by IDS.

Code copied to clipboard.
java -jar target/alfresco-elasticsearch-reindexing-4.0.0-app.jar \
  --alfresco.reindex.jobName=reindexByIds \
  --alfresco.reindex.pagesize=100 \
  --alfresco.reindex.batchSize=100  \
  --alfresco.reindex.fromId=1 \
  --alfresco.reindex.toId=10000 \
  --alfresco.reindex.concurrentProcessors=2

Sample invocation for Fetch by DATE.

Code copied to clipboard.
 java -jar target/alfresco-elasticsearch-reindexing-4.0.0-app.jar \
  --alfresco.reindex.jobName=reindexByDate \
  --alfresco.reindex.pagesize=100 \
  --alfresco.reindex.batchSize=100  \
  --alfresco.reindex.concurrentProcessors=6 \
  --alfresco.reindex.fromTime=202001010000 \
  --alfresco.reindex.toTime=202104180000

Alfresco Live Indexing app

The Alfresco Live Indexing app requires a working Alfresco ActiveMQ service, Alfresco Shared File store service, and the Elasticsearch server.

The table below lists the main configuration properties that can be specified through the Spring boot configuration.

Within the Elasticsearch connector there is a subset of components that index data. A component called Mediation subscribes to the channel indicated by the alfresco.event.topic attribute, as seen in the table above, and processes the incoming node events. The configuration of that component allows you to declare three blacklist sets for filtering out nodes or attributes to be indexed. These blacklists can be specified in the file using the alfresco.mediation.filter-file attribute, as seen in the table above. The default file is called mediation-filter.yml that must be in the module classpath, see the sample content of that file:

Code copied to clipboard.
mediation:
  nodeTypes:
     - nodeType1
     - nodeType2
     - ...
     - nodeTypeN
  contentNodeTypes:
     - nodeType1
     - nodeType2
     - ...
     - nodeTypeN
  nodeAspects:
     - nodeAspect1
     - nodeAspect2
     - ...
     - nodeAspectN
  fields:
     - field1
     - field2
     - ...
     - fieldN

Where:

• nodeTypes: if the node wrapped in the incoming event has a type which is included in this set, the node processing is skipped.
• contentNodeTypes: if the node wrapped in the incoming event has a content change associated with it and it has a type which is included in this set, then the corresponding content processing won’t be executed. This means nodes belonging to one of the node types in this set, won’t have any content indexed in Elasticsearch.
• nodeAspects: if the node wrapped in the incoming event has an aspect which is included in this set, the node processing is skipped.
• fields: fields listed in this set are removed from the incoming nodes metadata. This means fields in this set won’t be sent to Elasticsearch for indexing, and therefore they won’t be searchable.

To override some of these values command line system properties can be specified. Using the standard Spring boot approach, the name of the property must be converted to uppercase and dots must be changed by underscore characters. The following sample overrides the default values for three different properties.

Code copied to clipboard.
$ java -DSPRING_ELASTICSEARCH_REST_URIS=http://localhost:9200
 -DSPRING_ACTIVEMQ_BROKERURL=nio://activemq:61616
 -DALFRESCO_SHAREDFILESTORE_BASEURL=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file/
 -jar alfresco-elasticsearch-live-indexing-4.0.0-app.jar

The same convention can be used when deploying the Elasticsearch connector using the Docker compose template.

Code copied to clipboard.
live-indexing:
    image: quay.io/alfresco/alfresco-elasticsearch-live-indexing
    environment:
        SPRING_ELASTICSEARCH_REST_URIS: http://elasticsearch:9200
        SPRING_ACTIVEMQ_BROKERURL: nio://activemq:61616
        ALFRESCO_SHAREDFILESTORE_BASEURL: http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file/

For example, content indexing for cm:content documents can be disabled using the following Docker configuration:

Code copied to clipboard.
    live-indexing-mediation:
        image: quay.io/alfresco/alfresco-elasticsearch-live-indexing-mediation:${LIVE_INDEXING_MEDIATION_TAG}
        depends_on:
            - elasticsearch
            - alfresco
        environment:
            SPRING_ELASTICSEARCH_REST_URIS: http://elasticsearch:9200
            SPRING_ACTIVEMQ_BROKERURL: nio://activemq:61616
            ALFRESCO_MEDIATION_FILTER-FILE: file:/usr/tmp/mediation-filter.yml
        volumes:
            - ./mediation-filter.yml:/usr/tmp/mediation-filter.yml

The file mediation-filter.yml includes default content and also the cm:content filter.

Code copied to clipboard.
$ cat mediation-filter.yml
mediation:
  nodeTypes:
  contentNodeTypes:
    - cm:content
  nodeAspects:
    - sys:hidden
  fields:
    - cmis:changeToken

See Externalized Configuration for more.

Scaling up

All Elasticsearch connector services can be scaled up to use an ActiveMQ Connection Pool to increase the number of Consumers.

To use the ActiveMQ Connection Pool, add the following properties to your .env file in Docker compose. The Pool size is set to 100 in the sample.

Code copied to clipboard.
$ cat .env
ACTIVEMQ_POOL_ENABLED=true
ACTIVEMQ_POOL_SIZE=100

Spring related properties can be associated to the Elasticsearch connector when declaring the service in the docker-compose.yml file.

Code copied to clipboard.
    live-indexing-metadata:
        image: quay.io/alfresco/alfresco-elasticsearch-live-indexing-metadata:${LIVE_INDEXING_METADATA_TAG}
        environment:
            SPRING_ACTIVEMQ_BROKERURL: nio://activemq:61616
            SPRING_ACTIVEMQ_POOL_ENABLED: ${ACTIVEMQ_POOL_ENABLED}
            SPRING_ACTIVEMQ_POOL_MAXCONNECTIONS: ${ACTIVEMQ_POOL_SIZE}

To increase the consumer number you must check the property name in the application.properties file for the service and to then override it in the docker-compose.yml file. The following sample increases the consumer number to 20 for elasticsearch-live-indexing-metadata.

Code copied to clipboard.
    live-indexing-metadata:
        image: quay.io/alfresco/alfresco-elasticsearch-live-indexing-metadata:${LIVE_INDEXING_METADATA_TAG}
        environment:
            INPUT_ALFRESCO_METADATA_BATCH_EVENT_CHANNEL: sjms:org.alfresco.search.metadata.event?concurrentConsumers=20

Bulk deletion and ingestion

After the completion of a bulk deletion process, some scheduled jobs execute to clean up the environment for the next set of tasks. Executing a bulk ingestion process when the clean-up jobs are running impacts the performance of the ingestion process.

Therefore, when you plan to execute the bulk ingestion process after a bulk deletion process completes, it is recommended that you consider the following best practices:

• Start the bulk ingestion process after 25 minutes of the completion of the deletion process.
• Schedule the clean-up jobs after you complete the bulk ingestion.
• If you have configured the log files to include clean-up job notifications, monitor the log files and resume the ingestion after the cleanup is complete.

Using HTTP Basic Authentication to access Elasticsearch

When using the Elasticsearch server with the HTTP Basic Authentication protocol you must add your Elasticsearch credentials to the alfresco-global.properties configuration file.

Code copied to clipboard.
elasticsearch.user=elastic
elasticsearch.password=bob123

Additionally, for every “live-indexing” service from the Elasticsearch connector the same credentials must be configured. Use Java and the following global properties:

Code copied to clipboard.
SPRING_ELASTICSEARCH_REST_USERNAME=elastic
SPRING_ELASTICSEARCH_REST_PASSWORD=bob123

The environment variables can be passed as a command line argument when running the Spring boot application locally or they can be added to the environment service section when using Docker Compose. The example above connects to an Elasticsearch server configured with the following values:

Code copied to clipboard.
elasticsearch:
  image: elasticsearch:7.10.2
  environment:
    - discovery.type=single-node
    - xpack.security.enabled=true
    - ELASTIC_PASSWORD=bob123

You must also add these credentials to the Kibana app.

Code copied to clipboard.
kibana:
   image: kibana:7.10.1
   environment:
     - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
     - ELASTICSEARCH_USERNAME=elastic
     - ELASTICSEARCH_PASSWORD=bob123

Using HTTPS to access Elasticsearch for end to end encryption

When using the Elasticsearch server with the HTTPs protocol, additional configuration should be added to the Alfresco Repository.

A truststore file, including public certificate and certificate chain from Elasticsearch HTTPs endpoint, must be added to the alfresco-global.properties configuration file.

Add the following properties to use the truststore file from the Alfresco Repository. The encryption.ssl.truststore.passwordFileLocation= property has been intentionally left blank as the keystore includes only public information.

Code copied to clipboard.
encryption.ssl.truststore.location=/usr/local/tomcat/alf_data/keystore/truststore.jceks
encryption.ssl.truststore.passwordFileLocation=
encryption.ssl.truststore.type=JCEKS

If you are using Docker compose, add the same properties in the JAVA_OPTS section for the Alfresco service using the “-D” prefix.

Additionally, for every “live-indexing” service from the Elasticsearch connector the same truststore must be configured. Use Java and the following global properties:

Code copied to clipboard.
JAVAX_NET_SSL_TRUSTSTORE=/usr/local/tomcat/alf_data/keystore/truststore.jceks
JAVAX_NET_SSL_TRUSTSTOREPASSWORD=
JAVAX_NET_SSL_TRUSTSTORETYPE=JCEKS

These environment variables can be passed as command line arguments when running the spring boot application locally or they can be added to the environment service section when using Docker compose.

Exact Term Search

The Exact Term search feature, that allows searching using the equals operator =, is disabled by default to save index space. It’s possible to enable it for specific properties and property types using the following configuration file /alfresco/search/elasticsearch/config/exactTermSearch.properties located in the Alfresco Repository.

Note: Once you have the Exact term search configured a re-index is required. If you need the feature from the beginning, it is recommended to enable it before your first index is created.

You can add as many data-types and properties as you like by adding lines and incrementing the associated index:

Code copied to clipboard.
alfresco.cross.locale.datatype.0={http://www.alfresco.org/model/dictionary/1.0}text
alfresco.cross.locale.datatype.1={http://www.alfresco.org/model/dictionary/1.0}content
alfresco.cross.locale.datatype.2={http://www.alfresco.org/model/dictionary/1.0}mltext
alfresco.cross.locale.property.0={http://www.alfresco.org/model/content/1.0}content

In order to overwrite this configuration when using Docker, mount this file as an external volume. Following sample describes a local configuration to be applied to Elasticsearch Search Subsystem when using Docker Compose deployment:

Code copied to clipboard.
services:
  alfresco:
    volumes:
      - ./exactTermSearch.properties:/usr/local/tomcat/webapps/alfresco/WEB-INF/classes/alfresco/search/elasticsearch/config/exactTermSearch.properties

If you are using Content Services 23.3, refer to the following sample:

Code copied to clipboard.
services:
  alfresco:
    volumes:
      - ./exactTermSearch.properties:/usr/local/tomcat/shared/classes/alfresco/extension/exactTermSearch.properties

Support for different databases

PostgreSQL is the default database for Search Enterprise. You can use different databases with Search Enterprise, but they must be configured within your system and must match the database used by Content Services. The other types of databases supported by Search Enterprise are: MySQL, MariaDB, Microsoft SQL Server, and Oracle.

Add parameters to the startup script or in the command line when you run the reindexing app. You can use the following parameters.

For example:

Code copied to clipboard.
java -jar alfresco-elasticsearch-reindexing-4.0.0.1.jar --spring.datasource.url="jdbc:sqlserver://<your-server>.<your-domain>.com:1433;databaseName=alfresco;integratedSecurity=true"

Provide custom JDBC Drivers

Search Enterprise only provides the PostgreSQL driver by default and it is bundled with the Search Enterprise executable components. If you want to use a different database to PostgreSQL you must provide the correct JDBC configuration and corresponding driver. The drivers are loaded from a directory called db-drivers that must be present at the same directory level as the executable .jar file.

For example:

Code copied to clipboard.
├── `alfresco-elasticsearch-reindexing-x.x.x-app.jar`
└── `db-drivers`
    └── `mydb-driver.jar`

If you are using Docker Compose to install Search Enterprise you must add the JDBC driver information inside the docker container.

For example:

Code copied to clipboard.
services:
    reindexing-service:
        image: quay.io/alfresco/alfresco-elasticsearch-reindexing:latest
        mem_limit: 1024m
        environment:
        - ...
        volumes:
            - ./<location>/jdbc/drivers:/opt/db-drivers:ro