Integrate with Alibaba Cloud Monitor¶
What is Alibaba Cloud Monitor¶
Alibaba Cloud provides a fully managed, maintenance-free observability platform that enables one-click monitoring, tracing, and evaluation of FlexAI applications.
Info:
Alibaba Cloud Monitor natively supports Python/Golang/Java applications through LoongSuite agents and open-source OpenTelemetry agents. In addition to one-click monitoring of FlexAI LLM applications, it also supports end-to-end observability of FlexAI components and their upstream and downstream dependencies through non-invasive agents.
For more details, please refer to the Cloud Monitor documentation.
How to Configure Alibaba Cloud Monitor¶
1. Get Alibaba Cloud Endpoint and License Key¶
- Log in to the ARMS console, and click Integration Center in the left navigation bar.
- In the Server-side Applications area, click the OpenTelemetry card.
- In the OpenTelemetry panel that appears, select gRPC as the export protocol, and select the connection method and region according to your actual deployment.
- Save the Public Endpoint and Authentication Token (License Key).
Note:
The Endpoint does not include a port number, for example
http://tracing-cn-heyuan.arms.aliyun.com.
2. Configure Cloud Monitor in FlexAI¶
Info:
Prerequisites: FlexAI Cloud or Community Edition version must be ≥ v1.6.0
- Log in to the FlexAI console and navigate to the application you want to monitor.
- Open Monitoring in the left navigation bar.
- Click Tracing app performance, then click Configure in the Cloud Monitor area.
- In the dialog that appears, enter the License Key and Endpoint obtained in step 1, and customize the App Name (the application name displayed in the ARMS console), then click Save & Enable.
View Monitoring Data in Alibaba Cloud Monitor¶
After configuration, debug or production data from applications in FlexAI can be monitored in Cloud Monitor.
Method 1: Jump to ARMS Console from FlexAI Application¶
In the FlexAI console, select an application with tracing enabled, go to Tracing Configuration, and click View in the Cloud Monitor area.
Method 2: View Directly in ARMS Console¶
Go to the corresponding FlexAI application in the LLM Application Monitoring > Application List page of the ARMS console.
Access More Data¶
Cloud Monitor provides multi-language non-invasive agents that support accessing various components of the FlexAI cluster to achieve end-to-end tracing.
| FlexAI Component | Agent | Details |
|---|---|---|
| Nginx | OpenTelemetry Agent | Use OpenTelemetry for Nginx Tracing |
| API | LoongSuite-Python Agent | loongsuite-python-agent |
| Sandbox | LoongSuite-Go Agent | loongsuite-go-agent |
| Worker | OpenTelemetry Agent | Submit Python Application Data via OpenTelemetry |
| Plugin-Daemon | LoongSuite-Go Agent | loongsuite-go-agent |
Monitoring Data List¶
Cloud Monitor supports collecting data from FlexAI's Workflow/Chatflow/Chat/Agent applications, including execution details of workflows and workflow nodes, covering model calls, tool calls, knowledge retrieval, execution details of various process nodes, as well as metadata such as conversations and user information.
Workflow/Chatflow Trace Information¶
| Workflow | Alibaba Cloud Monitor Trace |
|---|---|
| workflow\_id | Unique identifier of the Workflow |
| conversation\_id | Conversation ID |
| workflow\_run\_id | ID of this run |
| tenant\_id | Tenant ID |
| elapsed\_time | Duration of this run |
| status | Run status |
| version | Workflow version |
| total\_tokens | Total tokens used in this run |
| file\_list | List of processed files |
| triggered\_from | Source that triggered this run |
| workflow\_run\_inputs | Input data for this run |
| workflow\_run\_outputs | Output data for this run |
| error | Errors that occurred during this run |
| query | Query used during runtime |
| workflow\_app\_log\_id | Workflow application log ID |
| message\_id | Associated message ID |
| start\_time | Run start time |
| end\_time | Run end time |
Workflow Trace Metadata
- workflow_id - Unique identifier of the Workflow
- conversation_id - Conversation ID
- workflow_run_id - ID of this run
- tenant_id - Tenant ID
- elapsed_time - Duration of this run
- status - Run status
- version - Workflow version
- total_tokens - Total tokens used in this run
- file_list - List of processed files
- triggered_from - Trigger source
Message Trace Information¶
| Message | Alibaba Cloud Monitor Trace |
|---|---|
| message\_id | Message ID |
| message\_data | Message data |
| user\_session\_id | User's session\_id |
| conversation\_model | Conversation model |
| message\_tokens | Number of tokens in the message |
| answer\_tokens | Number of tokens in the answer |
| total\_tokens | Total tokens in message and answer |
| error | Error information |
| inputs | Input data |
| outputs | Output data |
| file\_list | List of processed files |
| start\_time | Start time |
| end\_time | End time |
| message\_file\_data | File data associated with the message |
| conversation\_mode | Conversation mode |
Message Trace Metadata
- conversation_id - ID of the conversation to which the message belongs
- ls_provider - Model provider
- ls_model_name - Model ID
- status - Message status
- from_end_user_id - ID of the sending user
- from_account_id - ID of the sending account
- agent_based - Whether it is agent-based
- workflow_run_id - Workflow run ID
- from_source - Message source
- message_id - Message ID
Dataset Retrieval Trace Information¶
| Dataset Retrieval | Alibaba Cloud Monitor Trace |
|---|---|
| message\_id | Message ID |
| inputs | Input content |
| documents | Document data |
| start\_time | Start time |
| end\_time | End time |
| message\_data | Message data |
Dataset Retrieval Trace Metadata
- message_id - Message ID
- ls_provider - Model provider
- ls_model_name - Model ID
- status - Message status
- from_end_user_id - ID of the sending user
- from_account_id - ID of the sending account
- agent_based - Whether it is agent-based
- workflow_run_id - Workflow run ID
- from_source - Message source
Tool Trace Information¶
| Tool | Alibaba Cloud Monitor Trace |
|---|---|
| message\_id | Message ID |
| tool\_name | Tool name |
| start\_time | Start time |
| end\_time | End time |
| tool\_inputs | Tool inputs |
| tool\_outputs | Tool outputs |
| message\_data | Message data |
| error | Error information (if any) |
| inputs | Input content of the message |
| outputs | Answer content of the message |
| tool\_config | Tool configuration |
| time\_cost | Time cost |
| tool\_parameters | Tool parameters |
| file\_url | URL of associated file |
Tool Trace Metadata
- message_id - Message ID
- tool_name - Tool name
- tool_inputs - Tool inputs
- tool_outputs - Tool outputs
- tool_config - Tool configuration
- time_cost - Time cost
- error - Error information
- tool_parameters - Tool parameters
- message_file_id - Message file ID
- created_by_role - Creator role
- created_user_id - Creator user ID