Nov 15th, 2022

Observability - ApostropheCMS, OpenTelemetry, and New Relic

In this article, we will look at how to connect your site to New Relic using the OpenTelemetry observability framework.

Victor Jonah Community Contributor

What is OpenTelemetry?

OpenTelemetry, an open-source project, offers a set of vendor-neutral APIs that allow us to collect and transfer data to a location. This enables you to understand what your site is doing without having to worry about the tools or programming language being used. For more information, check out Open Telemetry’s website.

What is New Relic?

New Relic is a monitoring tool that can import data emitted by OpenTelemetry to help analyze and resolve application or website issues. It provides real-time data and shows the analytical performance of your site on the New Relic dashboard. It is designed to help discover unexpected outcomes and help you improve performance based on key metrics.

What is Observability?

Observability has become more important for sites in recent years. Most people tend to confuse Observability and Monitoring, but the main difference is that Monitoring tells you when something goes wrong, and Observability tells you why it went wrong. Monitoring has to happen before Observability takes place. Observability is one important concept that needs to be considered in complex systems, because if it doesn’t exist, then you can never know why things go wrong. This, in turn, will affect business performance and customer experience.

Profiling ApostropheCMS with OpenTelemetry already provides instructions for using another tool, Jaeger. The reason we are using New Relic in this article is that it provides more features out-of-the-box besides distributed tracing or monitoring. This article will be a quick guide on configuring this tool, which will extract telemetry data and export them to New Relic for monitoring.

Before moving on with this guide, we recommend you update Apostrophe to version 3.18.0, or preferably the latest 3.x version available.

Setup OpenTelemetry and NewRelic

Before we begin, there are a few things to note; we will be using an SDK for OpenTelemetry, which basically is used for measurement, and those measurements(data) will be exported to New Relic using an OTLP/PROTO Trace Exporter. The reason for using OTLP/PROTO is that New Relic expects its data in Proto format unlike the more familiar JSON format.

Step 1: Install the required dependencies for OpenTelemetry

Install the required dependencies in your project:

$ npm install @opentelemetry/sdk-node@0.27.0 @opentelemetry/auto-instrumentations-node@0.27.4 @opentelemetry/semantic-conventions@1.0.1 @opentelemetry/resources @opentelemetry/exporter-trace-otlp-proto uuid

With that complete, there are two ways we can export our data to New Relic; exporting directly to New Relic from our app or exporting to New Relic from a collector. But in this guide, we will be exporting directly from our Apostrophe app.

Step 3: Configure OpenTelemetry

At this point, we are about to do the real work where we have to configure OpenTelemetry and export telemetry data to New Relic. Exporting this kind of data relies on a specific protocol; the OpenTelemetry Protocol or OTLP.

In your project root directory, create a file telemetry.js:

const { NodeSDK } = require('@opentelemetry/sdk-node');
    const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
    const { SemanticResourceAttributes } = require('@opentelemetry/semantic-conventions');
    const { Resource } = require('@opentelemetry/resources');
    const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-proto');
    const { v4: uuidv4 } = require('uuid');
    const pkg = require('./package-lock.json');

    // Resources attributes
    const resource = new Resource({
      [SemanticResourceAttributes.SERVICE_NAME]: pkg.name,
      [SemanticResourceAttributes.SERVICE_VERSION]: pkg.version,
      [SemanticResourceAttributes.INSTANCE]: uuidv4(),
    });

    const instrumentations = [getNodeAutoInstrumentations()];

    const traceExporter = new OTLPTraceExporter({
      url: '<https://otlp.nr-data.net:4317/v1/traces>',
      headers: {
        'api-key': '<your-api-key>', // Your license key from NewRelic account
      },
    });

    const sdk = new NodeSDK({
      resource,
      instrumentations,
      traceExporter,
    });

    const shutdown = () => {
      sdk
        .shutdown()
        .then(() => {
          console.log('Shutdown complete');
        })
        .catch((err) => {
          console.error('Shutdown failed', err);
        });
    };

    module.exports = {
      sdk,
      shutdown,
    };

This code imports the necessary packages we had installed earlier and provides the resources attributes, which is more information about the service. It also enables auto-instrumentation (OpenTelemetry project facilitates the instrumentation or non-core components) before configuring the OTLP/PROTO trace exporter.

Finally, this code exports both the configured SDK function we will use and a shutdown function.

With that set, we have to refractor our app.js to make use of the opentelemetry setup:

const apostrophe = require('apostrophe');
    const process = require('process');
    const { sdk, shutdown } = require('./telemetry');

    const config = {
      shortName: 'apos-relic',
      modules: {},
    };

    if (process.env.APOS_OPENTELEMETRY) {
      sdk
        .start()
        .then(() => console.log('Started OpenTelemetry'))
        .then(() => apostrophe(config))
        .catch((err) => console.error('Failed to start OpenTelemetry', err));
    } else {
      apostrophe(config);
    }

    process.on('SIGTERM', () => {
      shutdown()
        .then(() => console.log('Tracing terminated'))
        .catch((error) => console.log('Error terminating tracing', error))
        .finally(() => process.exit(0));
    });

Three important observations you will need to look at on your New Relic UI are the traces, logs and metrics. Traces, otherwise known as distributed tracing, are basically application requests which are tagged with a unique identifier. Logs, on the other hand, are vital information and trigger alerts as they happen in your application. These may need some setup before any data is logged. Metrics are the core information or better still statistics; viewership statistics, engagement statistics, etc.

You may decide to trace command line tasks as well.

Liked this article? Feel free to share and subscribe