ElasticSearch provides downloads in multiple formats, however the ZIP download is recommended for general development usage. As mentioned above, you must use ElasticSearch 1.7.3 with the Java client embedded in Process Services. The same can be downloaded from the ElasticsSearch releases archive [7].

Once you have extracted this to a local directory, configure to ensure the ElasticSearch Java client is able to find the cluster in config/elasticsearch.yml. In general, you can leave the ElasticSearch default settings as-is.

network.host: 127.0.0.1

Now you can run ElasticSearch from a terminal, e.g.

./bin/elasticsearch

Or on Windows

bin/elasticsearch.bat

The logging output to the console should indicate that ElasticSearch is running successfully on ports 9200 and 9300, e.g.

[2016-03-15 16:43:50,117][INFO ][node                     ] [Bird-Brain] version[1.7.5], pid[2674], build[00f95f4/2016-02-02T09:55:30Z]
[2016-03-15 16:43:50,117][INFO ][node                     ] [Bird-Brain] initializing ...
[2016-03-15 16:43:50,171][INFO ][plugins                  ] [Bird-Brain] loaded [], sites []
[2016-03-15 16:43:50,198][INFO ][env                      ] [Bird-Brain] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [112.9gb], net total_space [464.7gb], types [hfs]
[2016-03-15 16:43:51,831][INFO ][node                     ] [Bird-Brain] initialized
[2016-03-15 16:43:51,832][INFO ][node                     ] [Bird-Brain] starting ...
[2016-03-15 16:43:51,922][INFO ][transport                ] [Bird-Brain] bound_address {inet[/127.0.0.1:9300]}, publish_address {inet[/127.0.0.1:9300]}
[2016-03-15 16:43:51,936][INFO ][discovery                ] [Bird-Brain] elasticsearch/Txeg7JYVTE6H4314aQEoZA
[2016-03-15 16:43:55,710][INFO ][cluster.service          ] [Bird-Brain] new_master [Bird-Brain][Txeg7JYVTE6H4314aQEoZA][MacBook-Pro.local][inet[/127.0.0.1:9300]], reason: zen-disco-join (elected_as_master)
[2016-03-15 16:43:55,729][INFO ][http                     ] [Bird-Brain] bound_address {inet[/127.0.0.1:9200]}, publish_address {inet[/127.0.0.1:9200]}
[2016-03-15 16:43:55,729][INFO ][node                     ] [Bird-Brain] started
[2016-03-15 16:43:55,751][INFO ][gateway                  ] [Bird-Brain] recovered [2] indices into cluster_state
[2016-03-15 16:44:41,023][INFO ][cluster.service          ] [Bird-Brain] added {[activiti-client][wKX2Tj1CR2-WesIRKKE1iQ][MacBook-Pro.local][inet[/127.0.0.1:9301]]{client=true, data=false},}, reason: zen-disco-receive(join from node[[activiti-client][wKX2Tj1CR2-WesIRKKE1iQ][MacBook-Pro.local][inet[/127.0.0.1:9301]]{client=true, data=false}])

Sense is a simple Kibana plugin which allows you to send requests to a running ElasticSearch node and see the results displayed in its console. It is especially helpful when you are developing your queries as it allows you to to try these out without any coding, but it also defines a basic cURL-like syntax for requests which is used in all the code examples in the ElasticSearch docs.

You must install the latest version of Kibana [8] in order to be able to use the Sense plugin. This version of Kibana supports ElasticSearch 1.7.x but in order to use it you must add the following lines to the end of your config/kibana.yml file in order to turn off the core Kibana functionality which will not work with older versions of ElasticSearch. This does not matter, since the Sense plugin runs standalone and connects to ElasticSearch directly.

kibana.enabled: false        # disable the standard kibana discovery, visualize & dashboard plugins
elasticsearch.enabled: false # do not require a running Elasticsearch 2.0 instance

After this we can install the Sense plugin and start Kibana

./bin/kibana plugin --install elastic/sense
./bin/kibana

You should see some information output to the console to indicate that Kibana is running and you can navigate to Sense by pointing your browser to http://localhost:5601/app/sense [9]

To test that everything is working and to start getting a feel for the data mappings defined in the Process Services indexes, enter the following query into the Sense UI and hit the green execute button.

GET _mapping 

For some events to show up in the Elasticsearch indexes, you must first turn on event generation and processing for your installation and also enable Elasticsearch itself.

To turn on event generation and processing, add the following to the tomcat/lib/activiti-app.properties file.

event.generation.enabled=true
event.processing.enabled=true

The most straightforward configuration for Elasticsearch is to use the embedded mode, which will start a new local, standalone node within the webapp itself. Don’t forget to turn on HTTP access also for development (do not do this in production!) so that you can use cURL or Sense to send queries to Elasticsearch directly.

elastic-search.server.type=embedded
elastic-search.enable.http=true

Alternatively you can install an external instance of Elasticsearch.

By default, ElasticSearch will use multicast to find other nodes. You configure ElasticSearch to use unicast to reach the local ElasticSearch node that you have started. The following configuration, added to the activiti-app.properties file, will configure client mode and unicast discovery in addition to turning on event generation and processing so that these get sent to ElasticSearch.

elastic-search.server.type=client
elastic-search.cluster.name=elasticsearch
elastic-search.discovery.type=unicast
elastic-search.discovery.hosts=127.0.0.1[9300-9400]

Once you have completed this setup you should restart Process Services to ensure that the settings are applied. The console logging output should contain some information indicating that the ElasticSearch client has successfully connected to the cluster.

A custom report is a custom section available in the Analytics app and also within each published app, which shows one or more custom reports.

Each report is implemented by a Spring bean which is responsible for two things:

1. Perform an ElasticSearch search query using the Java client API.
2. Convert the search results (hits or aggregations) into chart or table data and add this to the response.

The UI will automatically display the correct widgets based on the data that your bean sends.

Your Spring bean will be discovered automatically via annotations but must be placed under the package com.activiti.service.reporting. Since this package is used for the out-of-the-box reports it is recommended that custom reports use the sub-package such as com.activiti.service.reporting.custom.

The overall structure of the class will be as follows, for the full source please see the web link at the end of this section.

package com.activiti.service.reporting.custom;

import com.activiti.domain.reporting.ParametersDefinition;
import com.activiti.domain.reporting.ReportDataRepresentation;
import com.activiti.service.api.UserCache;
import com.activiti.service.reporting.AbstractReportGenerator;
import org.activiti.engine.ProcessEngine;
import org.elasticsearch.client.Client;
import org.springframework.stereotype.Component;

import java.util.Map;

@Component(CustomVariablesReportGenerator.ID)
public class CustomVariablesReportGenerator extends AbstractReportGenerator {

    public static final String ID = "report.generator.fruitorders";
    public static final String NAME = "Fruit orders overview";

    @Override
    public String getID() {
        return ID;
    }

    @Override
    public String getName() {
        return NAME;
    }

    @Override
    public ParametersDefinition getParameterDefinitions(Map<String, Object> parameterValues) {
        return new ParametersDefinition();
    }

    @Override
    public ReportDataRepresentation generateReportData(ProcessEngine processEngine,
                                                       Client elasticSearchClient, String indexName, UserCache userCache,
                                                       Map<String, Object> parameterMap) {

        ReportDataRepresentation reportData = new ReportDataRepresentation();

        // Perform queries and add report data here

        return reportData;
    }

You must implement the generateReportData() method which is declared abstract in the superclass, and you can choose to override the getParameterDefinitions() method if you need to collect some user-selected parameters from the UI to use in your query.

The generateReportData() method of your bean is responsible for two things:

• Perform one or more ElasticSearch queries to fetch report data

• Populate chart/table data from the query results

A protected helper method executeSearch() is provided which provides a concise syntax to execute an ElasticSearch search query given a query and optional aggregation, the implementation of which also provides logging of the query generated by the Java client API before it is sent. This can help with debugging your queries using Sense, or assist you in working out why the Java client is not generating the query you expect.

return executeSearch(elasticSearchClient,
                indexName,
                ElasticSearchConstants.TYPE_VARIABLES,
                new FilteredQueryBuilder(
                        new MatchAllQueryBuilder(),
                        FilterBuilders.andFilter(
                                new TermFilterBuilder("processDefinitionKey", PROCESS_DEFINITION_KEY),
                                new TermFilterBuilder("name._exact_name", "customername")
                        )
                ),
                AggregationBuilders.terms("customerOrders").field("stringValue._exact_string_value")
        );

The log4j configuration required to log queries being sent to ElasticSearch via executeSearch() is as follows

log4j.logger.com.activiti.service.reporting.AbstractReportGenerator=DEBUG

Alternatively you can manually execute any custom query directly via the Client instance passed to the generateReportData() method, for example:

return elasticSearchClient
                .prepareSearch(indexName)
                .setTypes(ElasticSearchConstants.TYPE_PROCESS_INSTANCES)
                .setQuery(new FilteredQueryBuilder(new MatchAllQueryBuilder(), applyStatusProcessFilter(status)))
                .addAggregation(
                        new TermsBuilder(AGGREGATION_PROCESS_DEFINITIONS).field(EventFields.PROCESS_DEFINITION_ID)
                                .subAggregation(new FilterAggregationBuilder(AGGREGATION_COMPLETED_PROCESS_INSTANCES)
                                        .filter(new ExistsFilterBuilder(EventFields.END_TIME))
                                        .subAggregation(new ExtendedStatsBuilder(AGGREGATION_STATISTICS).field(EventFields.DURATION))));

Generating chart data from queries can be accomplished easily using the converters in the com.activiti.service.reporting.converters package. This avoids the need to iterate over returned query results in order to populate chart data items.

Initially two converters AggsToSimpleChartBasicConverter and AggsToMultiSeriesChartConverter are provided to populate data for pie charts (which take a single series of data) and bar charts (which take multiple series) respectively. These two classes are responsible for iterating over the structure of the ES data, while the member classes of com.activiti.service.reporting.converters.BucketExtractors are responsible for extracting an actual value from the buckets returned in the data.

ReportDataRepresentation reportData = new ReportDataRepresentation();

PieChartDataRepresentation pieChart = new PieChartDataRepresentation();
pieChart.setTitle("No. of orders by customer");
pieChart.setDescription("This chart shows the total number of orders placed by each customer");

new AggsToSimpleChartBasicConverter(searchResponse, "customerOrders").setChartData(
        pieChart,
        new BucketExtractors.BucketKeyExtractor(),
        new BucketExtractors.BucketDocCountExtractor()
);

reportData.addReportDataElement(pieChart);

SingleBarChartDataRepresentation chart = new SingleBarChartDataRepresentation();
chart.setTitle("Total quantities ordered per month");
chart.setDescription("This chart shows the total number of items that were ordered in each month");
chart.setyAxisType("count");
chart.setxAxisType("date_month");

new AggsToMultiSeriesChartConverter(searchResponse, "ordersByMonth").setChartData(
        chart,
        new BucketExtractors.DateHistogramBucketExtractor(),
        new BucketExtractors.BucketAggValueExtractor("totalItems")
);

reportData.addReportDataElement(chart);

For more details see the full source on the activiti-custom-reports [11] GitHub project.