Morio Client Guide

The Morio client forms an integral part of a Morio deployment. It gathers and ships data from your systems to your Morio cluster.

To do so, it leverages various agents from the Beats family, which you may already be familiar with.

However, it does more than that, and this guide will give you a good understanding of the Morio client and of the abstractions it provides, so that you can get the most value from it.

High level overview

As we mentioned, the Morio client uses various Beats agents to collect and ship data. However, assuming it’s just Beats — and potentially using it like this — will not be the best experience, as the client provides higher-level abstractions to help you manage clients at scale.

So before we dive into the details, let us take a moment to ensure we have a good mental model of what the client is and does. The schematic overview below will help us get there:

Rather than explain everything in this diagram, we will refer to it in this guide as we make progress.

Low level: Just Beats

The Morio client bundles various observability agents based on the Beats framework by Elastic. When you install the Morio client, the various Beats agents will be installed on your system as well, and you will be able to interact with them directly if you so wish.

Strictly speaking, you don’t even need the Morio client. You can install the Beats you want, and manually configure them to send data to Morio.

Already using Beats?

If you are already using Beats on a system, you can still install the Morio client. We make sure to not use any of the various Beats default configuration or data locations, so the Morio client can run side by side with your existing use of Beats agents.

Furthermore, while we do setup the Elastic software repository in the morio-repo package to ensure the various Beats dependencies can be installed, we configure it with a lower than default priority. So if you are installing Beats from a different software repository, Morio will use that repository.

Mid level: Local module templates and vars

The various Beats agents have — apart from their base configuration — a bunch of modules and/or inputs (the terminology depends a bit on the type of agent), which determine what data will be collected on the system.

Managing this at scale gets quickly tiresome, so this is where Morio adds the first abstraction by using module templates and vars.

Morio client templates

A Morio client template is a modular piece of agent configuration. It can describe what log files to collect (for Filebeat), what files to watch for changes (for Auditbeat), or what metrics to collect (for Metricbeat), and so on. As we like to keep things simple, we bundle templates for the various agents into a single module by naming convention. Then, the Morio client can enable or disable such a module, which means that rather than having to manage 3 individual agents, you have one single integrated configuration.

We maintain a curated list of templates on Moriohub. We consider this a communal effort to take the drudge out of configuring observability. For example, why does everyone running an NGINX web server have to come up with their configuration? They shouldn’t, not when you can use a template instead, and just enable the linux-nginx module (as an example).

If at this point you are thinking But you don’t know where I store my NGINX logs!, then thank you for providing the perfect segue to our next point.

Morio client vars

Invariably, no template can work out of the box for everyone. Perhaps logs are stored in a different location. Maybe on your production systems you are looking to collect more metrics than on your testing systems, and so on.

That’s where Morio client vars come in. Template authors can use a variable rather than hard-code these settings. Furthermore, they can define a default for the variable. If that default works for you, great. If not, you merely need to set the variable to override the default.

You can set these variables with the Morio client, or you can use any automation tool you like because these vars are just files. The filename is the name of the variable, and its contents its value. Because you know, we like to keep it simple.

High level abstraction: Centralised client management

Having module templates and vars already makes it a lot easier to maintain a library of agent configurations that you can reuse across your systems. But it does not help with distributing these module templates, or managing these vars in a centralised way.

It is certainly possible to leverage some automation tool, be it Ansible or whatnot, to take on this task, but it’s a chore.

This is why the Morio client offers another level of abstraction that allows you to centrally manage both the client’s module templates and the vars.

In this scenario, the Morio Cluster stores information about the various module templates available, as well as their default vars. In addition, it can also keep track of host-specific vars that override these defaults.

The end result is that a client can pull the configuration from the cluster that provides centralised management.

Clients can do more than pull their configuration, we’ll cover that in the next section.

Morio data flows

We already knew that the client bundles agents that ship their data to the Morio Cluster. Now we’ve also learned that there’s a possibility to pull the configuration, which begs the question, how does this all work?

The diagram below outlines the different data flows specific to the client:

Legend

• Line animation:
  • Unidirectional data flow
  • Bidirectional data flow
• Line width:
  • Always-on data flow
  • Optional data flow (only when client listener is enabled)
• Connections:
  • Client to Cluster: tcp:443^https & tcp:9092^kafka
  • Cluster to git hosts tcp:443^https
    Note: You can configure git hosts on a non-default HTTPS port or over plain HTTP, but HTTPS is typical

Data plane

The data plane is relatively simple:

• Agents on the system running the Morio Client send data to the Kafka broker on the Morio Cluster. In the world of Kafka and streaming data, this is referred to as producing data. Each agent produces to a topic that is tied to the agent type:
  • Auditbeat produces to the audit topic
  • Filebeat produces to the logs topic
  • Metricbeat produces to the metrics topic
• The Morio client binary sends data about the system it is running on to the Morio Cluster.
  Note: This is initiated via the morio join and morio report client commands.

Control plane

The control plane only comes into play when using centralised client management. In that scenario, these are the additional data flows:

• The core service will preseed client templates from the configured Git providers and store them in the db service.
• The client can use the morio pull command to load its configuration from the Morio cluster via the API service.
• The client can use the morio push command to send its local configuration to the Morio cluster via the API service.
• When running the client listener:
  • A Morio operator can leverage the api service to issue commands to one, several, or all clients.
  • The api service will generate a unique command ID and store it in the db service.
  • The api service will then produce a message in the clients topic of the broker service that holds all the information about the command and the unique command ID.
  • The client listener will consume the message from the clients topic, it will run the requested command and report back to the api service about its progress and results for the given command ID.

The Morio client does not listen on any ports

The Morio client’s listener does not listen on any TCP ports. Instead, it daemonises and connects to the broker service where it listens for (consumes) events in the clients topic.

This, in combination with the Morio client’s access to the api service establishes a two-way control plane between cluster and client, without the client actually listening for any connections.

The morio binary

The Morio client will install the morio binary on your system.

You can try it out yourself by running:

sudo morio

sudo required

Since the Morio configuration is only accessible to the root user — and the client needs to be able to manipulate these files — you should precede all commands with sudo.

We won’t include sudo in every example, but please keep it in mind.

See the client reference documentation

For a complete overview of all client capabilities, please refer to the Morio client reference documentation.

Help and version info

To get help on how to use the client, run:

morio help

To get help on a specific command, use the --help or -h flag.

To get the version of the client, run:

morio version

Agent pass-through

You can invoke any of the agents through the client:

Auditbeat

morio audit

Filebeat

morio logs

Metricbeat

morio metrics

Doing so invokes the relevant Beat with the correct configuration and data paths.

As an example, to test the Filebeat output, you can run:

morio logs test output

Please refer to the relevant Beats documentation or to the inline help for more info.

Available Commands: reset Reset the client configuration restart Restart agents start Start agents status Shows agents status stop Stop agents

Modules, templates, and vars

Modules

You can manage the list of (locally) enabled modules with the modules command.

To list the currently enabled modules:

morio modules list

To enable a module:

morio modules enable linux-system

Vars

To set a variable, use the vars command:

morio vars set MY_VAR "Some value"

To show all vars, use:

morio vars list

Templates

Modules are made up of one or more templates, which can use one or more vars.

To gather all your modules and vars and turn them into a valid agent configuration, run the template command:

morio template

Client enrolment

To enrol a client, you join it to a cluster, passing it the cluster FQDN:

morio join my.cluster.morio.it

To enable a central module, you may want to first list all the modules available on the cluster:

morio modules list-remote

Then perhaps enable a given module:

morio modules enable-remote linux-system

Once the cluster knows what modules to assign to this client, you can pull the configuration:

morio pull

If you’ve made local changes to vars or enabled modules you want to store centrally, use the push command:

morio push