Introduction

OpenSearch is built to provide highly scalable and flexible search functionalities for various applications. It encompasses full-text search, log analytics, and real-time monitoring, presenting strong tools for data examination and visualization. The effective log management offered by OpenSearch is vital for diagnosing issues and monitoring applications.

The DigitalOcean App Platform allows for the forwarding of OpenSearch logs. You can route runtime logs from your App Platform applications to an OpenSearch cluster, enabling you to analyze and query your application logs using OpenSearch’s APIs and dashboards.

This method is particularly useful if you aim to:

• Monitor and Troubleshoot: Observe application performance and errors through the analysis of real-time logs.
• Analyze Performance: Acquire insights into web traffic trends and server metrics.
• Centralized Logging: Consolidate logs from various applications hosted on App Platform into a single OpenSearch instance for simpler management.

This tutorial guides you on how to set up your App Platform application to send logs to a DigitalOcean managed OpenSearch dashboard.

Prerequisites

Ensure you have the following before starting:

• A valid DigitalOcean account. For detailed guidance, refer to the Getting Started with DigitalOcean documentation.
• A Managed OpenSearch Cluster.
• An application operational on DigitalOcean App Platform. Discover how to create applications on App Platform by consulting DigitalOcean’s official documentation.

Step 1 – Creating Managed OpenSearch DB

To set up a managed OpenSearch database, navigate to the database section of the control panel and click on Create Database.

Select a datacenter/region. If you have multiple VPCs, opt for the preferred VPC network within the chosen region. Choose OpenSearch as your database engine.

Now, select your database’s configuration.

You will have the flexibility to modify the configuration and scale up any time after the cluster creation; however, downscaling is currently not possible.

OpenSearch also provides scalable storage; hence, you can decide on the storage size.

Lastly, assign a Name to the database cluster, select your desired Project, and include tags if necessary.

When finished, click on Create Database Cluster.

Step 2 – Configuring the Application

Logs can be forwarded to your Managed OpenSearch cluster for viewing in the OpenSearch Dashboard within the control panel.

To set up log forwarding, go to the Apps section of the control panel, select your app, and click on the Settings tab. In the Log Forwarding segment, click Edit and choose Managed OpenSearch.

Next, provide the necessary values:

Define a Destination name that will help you distinguish between multiple log integrations.

Select the DigitalOcean OpenSearch database. Upon clicking, a dropdown list of your available Managed OpenSearch clusters will appear. Choose the one suitable for your needs.

An Index name denotes a unique identifier for an index, helping in storing and organizing log data for efficient searching, analyzing, and visualizing logs according to established naming conventions.

Now, select the resource service from the App for which you intend to forward logs. To save this configuration, click on Add Log Destination.

Step 3 – Set Up OpenSearch Dashboard

In the console, on the OpenSearch Database page, click on OpenSearch Dashboard (found at the top right corner).

The Dashboard will prompt for login details. Your login credentials are accessible in the Connection Details section.

After logging into the dashboard, navigate from the top left menu to Management -> Dashboard Management.

There, select Index patterns. Index patterns are essential for retrieving data from OpenSearch.

• Select the Data source.
• You should see the Index name you specified during the logging configuration on your App Platform.
• Indices should include a timestamp field (commonly used in log data) for time-based queries and visualizations. You need to select the relevant field.
• Click Create index pattern to finalize the setup.

From the Menu -> OpenSearch Dashboards -> Discover, you should be able to view logs under the Index name you defined previously. If multiple indexes exist, you can select the desired one from the dropdown.

Conclusion

You’ve effectively configured your App Platform service to gather and forward logs to a DigitalOcean Managed OpenSearch database. You have also gained knowledge on creating a new dashboard, adding visualizations, and customizing your OpenSearch Dashboards. With these skills, you can now proficiently monitor and analyze your web server logs.

Thanks for learning with the DigitalOcean Community. Explore our services for compute, storage, networking, and managed databases.

Learn more about our products