Query Engines
Elasticsearch

Customer facing dashboards using Elasticsearch

Get started

To get build your customer facing dashboard with Vizzly and Elasticsearch, you can use the Vizzly Command Line Interface (CLI) and execute the following commands, or alternatively you can follow the guide on the Vizzly app here, after signing in to your account (opens in a new tab).

First, ensure the latest version of the Vizzly CLI is installed
npm i -g @vizzly/cli
Second, generate the secrets

These will be used in your identity setup at a later stage. You can find more information about the secrets on our security page, and more information regarding the identity setup on our identity prop page.

vizzly create-key-pair
Lastly, create the Vizzly config
vizzly init-config -i elasticsearch

You should now find you have the Vizzly CLI installed, with public and private .pem files, and a vizzly.config.json file which looks like the following;

{
  "dataSets": [],
  "publicKeyCertificates": [
    "<< Your public key >>"
  ],
  "connection": {
    "client": "elasticsearch",
    "node": "{{VIZZLY_ELASTICSEARCH_NODE}}",
    "auth": {
      "apiKey": "{{VIZZLY_ELASTICSEARCH_API_KEY}}"
    }
  }
}
See the connection type definition
{
  client: "elasticsearch";
  cloud?: {
    id: string;
  },
  auth:
    | { username: string; password: string; }
    | { apiKey: string; }
    | { id: string; api_key: string; }
    | { bearer: string; }
}
Environment variables

You will notice that the vizzly config we have just generated, contains mustache values such as {{VIZZLY_ELASTICSEARCH_CLOUD_ID}} and {{VIZZLY_ELASTICSEARCH_API_KEY}}. You have the choice of replacing these with the literal values, or setting these values as environment variables when you deploy your query engine.

Elasticsearch connection environment variables

VIZZLY_ELASTICSEARCH_CLOUD_ID

Set the cloud ID to use in the connection to your elasticsearch database.

To use this value in your Vizzly config file, set the value of the field to be "{{VIZZLY_ELASTICSEARCH_CLOUD_ID}}".

VIZZLY_ELASTICSEARCH_API_KEY

Set the API key to use in the connection to your elasticsearch database.

To use this value in your Vizzly config file, set the value of the field to be "{{VIZZLY_ELASTICSEARCH_API_KEY}}".

VIZZLY_ELASTICSEARCH_NODE

Set the node value to use in the connection to your elasticsearch database.

To use this value in your Vizzly config file, set the value of the field to be "{{VIZZLY_ELASTICSEARCH_NODE}}".

VIZZLY_ELASTICSEARCH_USERNAME

Set the elasticsearch username to use in the connection to your elasticsearch database.

To use this value in your Vizzly config file, set the value of the field to be "{{VIZZLY_ELASTICSEARCH_USERNAME}}".

VIZZLY_ELASTICSEARCH_PASSWORD

Set the elasticsearch password to use in the connection to your elasticsearch database.

To use this value in your Vizzly config file, set the value of the field to be "{{VIZZLY_ELASTICSEARCH_PASSWORD}}".

VIZZLY_ELASTICSEARCH_API_KEY_ID

Set the elasticsearch API key ID to use in the connection to your elasticsearch database.

To use this value in your Vizzly config file, set the value of the field to be "{{VIZZLY_ELASTICSEARCH_API_KEY_ID}}".

For secret values such as passwords and API keys, we strongly recommend using the mustache templates and provide their values by setting environment variables.

Data sets

You will notice that the dataSets key in your Vizzly config is empty. The next stage of the setup process is to define the schema of the data sets that you want to make available to your users.

The Elasticsearch data set structure is;

 
{
  /*
    ID of the data set that remains constant.
  */
  id: string;
 
  /*
  A public name that describes this data set to your users.
  */
  name: string;
 
  /*
  Fields that define the flat schema of the data set.
  */
  fields: Array<{
  /*
  A unique reference to this field, and should remain constant.
  */
  id: string;
 
  /*
    The name of the field. For example, this would be the column
    name if you're running a postgres database.
  */
  name: string;
 
  /*
    What type of data does this field contain?
  */
  dataType: "number" | "boolean" | "string" | "date_time";
 
  /*
    The name of this field that your users will see.
  */
  publicName: string;
 
  /*
    The name of the table in the database
  */
  table: string;
 
}>;
}

Data types

These are the valid data types supported for the Elasticsearch integration;

  • date_time
  • number
  • string
  • boolean

Secure filter guards (optional)

Secure filters are used to provide Vizzly's multi-tenancy feature, and they are defined in the identity config. As a safe-guarding measure to ensure these filters are always set when queries run against the data set, you can setup secureFilterGuards.

For example, if you had a data set which stored bank transactions, and you have an account_id field which you're using to determine access for your multi-tenancy platform, you will be setting a secure filter in your identity config such as account_id = account-abc. To ensure that this filter is always set, you can define the secure filter guards on the data set in your Vizzly config that ensures an = filter is always applied on the account_id field.

You would achieve this by defining the following secure filter guard;

{
  "fieldId": "account_id",
  "op": "="
}

The typescript definition of a secure filter guard is as follows:

type Operator =
  | ">"
  | "<"
  | "="
  | "!="
  | ">="
  | "<="
  | "is_one_of"
  | "is_not_one_of"
  | "starts_with"
  | "ends_with"
  | "contains_substring"
  | "does_not_contain_substring";
 
{
  fieldId: string; // The `id` of the field
  op: Operator;
}

Deployment

You can deploy your Vizzly query engine anywhere that can run a docker image, provided it can connect securely to your Elasticsearch instance.

We have documentation for deploying to Amazon Web Services (AWS), Google Cloud Platform (GCP), Heroku and locally for development and testing.