AutoInsights is a Python-based tool that provides automatic conformance checking insights for your Celonis data models. The software is entirely command-line based and allows you to choose which insights should be computed for the event logs of your choice. The output of the insights can be used to optimize your existing process models, make conformance checking decisions and gain a better understanding of internal processes captured by the analyzed event log.
If you have little to no experience with Python please follow the steps outlined in the next section, Requirements.
If you wish to know which insights are available you can check out the Featured Insights section of the README further down below.
In case you run into issues please check out the FAQ at the bottom of the README that contains an overview of frequently asked questions.
In order to use this tool you need to have a Celonis account which you can set up here.
Once you have logged into Celonis you need to generate an API key.
This can be done by clicking your user profile in the bottom left once you have logged in and navigating
to Edit Profile > API-Keys and
clicking the Create API Key button.
Overview of how to generate an API key in Celonis.
Before you can use the tool you need to setup a data model on your Celonis account that contains an activity table.
If you are not sure how to do this you can follow the steps outlined in the Data Integration documentation of Celonis.
This project requires Python 3.10 or later to be installed on your machine. If you need help installing Python, visit the offical Python 3 installation guide.
After installing Python you should be able to use the pip install command from the Shell/Terminal/Console
environment of your platform.
This project requires a number of external packages to run, below we detail two ways to make sure those packages are available when running the software:
You can install all needed packages by utilizing the included requirements.txt and running
pip install -r /PATH_TO_PROJECT_ROOT/requirements.txt
This project requires the pycelonis package to build. Please make sure this package is available in your
environment. You can install it using pip by running
pip install --extra-index-url=https://pypi.celonis.cloud/ pycelonis
In addition to pycelonis the project also uses the following non-standard packages:
pm4py
pycelonis
pandas
pytest
fake_useragent
selenium
undetected_chromedriver
You can install them by using the standard Python package installer and running
pip install NAME_OF_PACKAGE
The option to retrieve LLM feedback is available for the declare model insight and requires an OpenAI account. If you wish to make use of this feature you also need the following web browsers installed depending on which OS you are using:
Windows/Linux:
Chrome 126.0 or newer
macOS:
Safari 17.x or newer (Requires macOS 12.6 or later)
You can download the latest release of AutoInsights here.
Once you have fulfilled the requirements and downloaded the project files you can start the tool by running main.py in the root directory.
This can be done with a variety of tools:
- Interactive Mode
- Command Line
- Text Editors, e.g. VSCode
- IDEs, e.g. PyCharm
Once you have started execution you will be asked to configure a number of parameters to retrieve a data model from your Celonis account and specify which insights you want to compute.
In total you need to specify 5 parameters before any insights can be computed:
Base URL: The base URL is the URL linking to the landing page of your Celonis account and usually ends
with [...]celonis.cloud.
API Token: This is a long string of characters making up the API token that you previously generated and is associated
with your Celonis account.
Example of initial Startup for the tool.
After entering the Base URL and API Token the software will give you an overview of the contents of your
account to make it easier to specify the last 3 parameters:
Data Pool name: The name of the data pool containing the relevant data model.
Data Model name: The name of the data model in the data pool that contains the relevant event log.
Insights: The insights that you wish to compute, you can enter the name of the insight or its associated number.
Example of choosing the Data Pool, Data Model and the Temporal Profile as an insight to compute.
The software will then start with the computation of the insights.
If you want to adjust your configuration you can do so by deleting or editing the service_config.ini located
in .../Your_Project_Folder/celonis_service.
We also provide all required Docker files in case you want to run the application in a Docker container. You can create the image by executing
docker build -t "autoinsightsimage" .
in the root directory of the project. This will create an image with the name autoinsightsimage based on the Dockerfile. To start a container using this image run
docker run -it autoinsightsimage
which will start a new container containing the application data. To specify a container name use the --name flag in the command. You can also connect to a running container instance by executing
docker exec -it NAME_OF_CONTAINER ENVIRONMENT_NAME
which will build a connection to a container using the given environment, e.g. bash.
The tool offers several insights that you can choose from based on your needs for the event log you want to analyze.
We have implemented an approach to computing the temporal profile as introduced by Stertz, Florian, Jürgen Mangler, and Stefanie Rinderle-Ma in the paper "Temporal Conformance Checking at Runtime based on Time-infused Process Models"(2008) that is available here.
The temporal profile utilizes the timestamps of an event log to calculate the average waiting time and standard deviation time for a pair of activities in the event log. For event logs with an end timestamp, the measurement is extended by an analysis of the lifetime/lifecycle of those activities.
The computation of the temporal profile is divided into three steps in total. First, a basic temporal profile is computed containing the average waiting and standard deviation time. Based on the results obtained, a z-score is then calculated in the second step. Lastly the deviation cost is computed by discovering a petri net and performing a token-based replay. During each step, the intermediate results along with some helpful metrics will be displayed to provide some level of intuition.
The z-score and deviation cost can be used to evaluate the performance and efficiency of specific activities that occur in the event log.
Below is an example of the resulting temporal profile for an event log that contains end timestamps:
During the computation you are able to specify a minimum threshold for the z-score and deviation cost to filter your
results. You additionally have the option to save the temporal profile as a .csv file if you wish to continue to work with
the resulting table.
Multiple options are available for users to get insight for conformance checking.
Activities repeated by different resources: Identifying activities that are performed by multiple resources. Example of resulting outcome:
Segregation of duties: Ensuring that certain activities are performed by different resources to avoid conflicts of interest. Based on the example Segregation of Duties, user is expected to choose the activities where to check SoD and the outcome is the violated list. Example of resulting outcome:
Resource-activity performance: Analyzing the performance of resources by checking the throughtput of resources in executing activities. Example of resulting outcome:
Identification of batches: Based on the paper Batch Processing Definition and Event Log Identification, it determines sequential/simultaneous/concurrent activities by same resources for different caseid. It expects user to input the threshold days for sequential batch identification. Example of resulting outcome:
We have implemented the computation of the DECLARE model as proposed by Fabrizio M. Maggi, Arjan J. Mooij, Wil M.P. van der Aalst in the paper "User-guided discovery of declarative process models" (2011) that is available here
It consists of a set of constraints based on templates, where templates define parameterized classes of properties, and constraints are concrete instantiations of these templates. The DECLARE language is designed to be understandable for end-users while also having a formal semantics for verification and execution purposes. Each constraint in a DECLARE model inherits the graphical representation and semantics from its template, making it user-friendly and verifiable.
Below is an example of the resulting DECLARE for an event log with 'existence', 'response', 'succession' chosen as templates:
Below you can find a small description of the contents of the project:
/main.py file: Start execution of the software, this is the file that should be run to start the tool.
/celonis_service folder: Contains the functionality related to setting up the celonis service which retrieves
objects
from the Celonis platform.
/insights folder: Contains the functionality of the different insights that can be used on a Celonis data model.
/misc folder: Contains various helper classes and functions that are used in other places of the project.
/output folder: This folder will contain the output of your computations after execution has stopped.
/resources folder: Contains files not relevant for software execution such as images for README.md.
/testing folder: Contains the pytest framework for the project and tests meant to test various functionality of the tool.
/requirements.txt file: Contains the names and versions of required external libraries to run the project.
/compose.yaml file: Contains information on how the Docker application stack and its services should look like.
/Dockerfile file: Contains the configuration for building the Docker image.
/.dockerignore file: Contains information on which files should appear in the built Docker image.
/.gitlab-ci.yml file: Contains the configuration of the CI/CD pipeline of the repository.
Below we list some questions that have occured during development and their respective answers that might help you with specific issues.
The project was developed using Python 3.10. However, you should also be fine just sticking to the latest version of Python since releases are backwards compatible. We would still recommend not changing the Python version in the Dockerfile if you plan to deploy the project in a container just to be safe since Docker is a bit unstable when it comes to Python-based projects and this might introduce unforeseen issues.
I am unable to access the Data Integration feature in Celonis/ I can't create data models in my data pool.
Please make sure you have either an academic or paid Celonis license. Free Celonis accounts are heavily limited in the number of features available in the Celonis workspace.
Docker only supports headless selenium drivers since there is no browser GUI available within containers. The webservice component does not work in a headless browser instance and as such LLM feedback is not available when computing templates with the declare model, attempting to use this option will result in an error.
I want to use an event log with an end timestamp for the temporal profile but the insight returns an empty dataframe, why is that?
When computing the temporal profile, the insight will check whether the current activity table contains an end timestamp column. If you setup your activity table in the Celonis UI without the usage of PQL then there is a high chance Celonis will initiate the end_timestamp attribute for your data model as "None", even when multiple columns contain timestamp values. If your activity table has an end timestamp , make sure that the start timestamp is set as the timestamp column and manually enter the name of the end timestamp column when prompted to do so. Alternatively you can create an event log with PQL using the CREATE_EVENTLOG operator as described here.
It is possible to calculate the DECLARE model using all available templates, although this is not recommended due to the increased computation times. It is generally advisable to only compute a small subset of the templates at a time.
The lengthy processing time is due to the way Celonis handles PQL queries. While there is no immediate solution, we are working on new and improved version to deal with this issue.
If multiple resource columns are detected, the method will list them and prompt you to select the correct one. You need to enter the corresponding number for the resource column you want to use.
Ensure your input matches the expected format. For example, when selecting activities by their numbers, provide a comma-separated list of valid numbers. If invalid input is detected, the method will prompt you again or use default values where applicable.
You can pass the necessary parameters directly to the methods (e.g., to_file, selected_indices, threshold_input, save_output) to avoid interactive prompts and ensure smooth execution in automated environments.
Ensure your input matches the expected format. For example, when selecting activities by their numbers, provide a comma-separated list of valid numbers. If invalid input is detected, the method will prompt you again or use default values where applicable.