Semantic conventions for RPC spans
Status: Development
This document defines how to describe remote procedure calls (also called “remote method invocations” / “RMI”) with spans.
Existing RPC instrumentations that are using v1.37.0 of this document (or prior):
- SHOULD NOT change the version of the RPC conventions that they emit by default in their existing major version. Conventions include (but are not limited to) attributes, metric and span names, and unit of measure.
- SHOULD introduce an environment variable
OTEL_SEMCONV_STABILITY_OPT_INin their existing major version as a comma-separated list of category-specific values (e.g., http, databases, rpc). The list of values includes:rpc- emit the stable RPC conventions, and stop emitting the experimental RPC conventions that the instrumentation emitted previously.rpc/dup- emit both the experimental and stable RPC conventions, allowing for a phased rollout of the stable semantic conventions.- The default behavior (in the absence of one of these values) is to continue emitting whatever version of the old experimental RPC conventions the instrumentation was emitting previously.
- Note:
rpc/duphas higher precedence thanrpcin case both values are present
- SHOULD maintain (security patching at a minimum) their existing major version for at least six months after it starts emitting both sets of conventions.
- MAY drop the environment variable in their next major version and emit only the stable RPC conventions.
Common remote procedure call conventions
Span name
RPC spans MUST follow the overall guidelines for span names.
The span name SHOULD be {rpc.method} if it is available and not set to
_OTHER.
If rpc.method is unavailable or set to _OTHER, the span name SHOULD be
{rpc.system.name}.
Semantic conventions for individual RPC systems MAY specify different span name format.
RPC client span
Status:
This span represents an outgoing Remote Procedure Call (RPC).
RPC client spans SHOULD cover the entire client-side lifecycle of an RPC, starting when the RPC is initiated and ending when the response is received or the RPC is terminated due to an error or cancellation.
For streaming RPCs, the span covers the full lifetime of the request and/or response streams until they are closed or terminated.
If a transient issue happened and was retried within this RPC, the corresponding span SHOULD cover the duration of the logical call with all retries.
Span name: refer to the Span Name section.
Span kind MUST be CLIENT.
Span status Refer to the Recording Errors document for details on how to record span status.
Attributes:
| Key | Stability | Requirement Level | Value Type | Description | Example Values |
|---|---|---|---|---|---|
rpc.system.name |  | Required | string | The Remote Procedure Call (RPC) system. [1] | grpc; dubbo; connectrpc |
error.type |  | Conditionally Required If and only if the operation failed. | string | Describes a class of error the operation ended with. [2] | DEADLINE_EXCEEDED; java.net.UnknownHostException; -32602 |
rpc.method | | Conditionally Required if available. | string | The fully-qualified logical name of the method from the RPC interface perspective. [3] | com.example.ExampleService/exampleMethod; EchoService/Echo; _OTHER |
rpc.method_original | | Conditionally Required If and only if it’s different than rpc.method. | string | The original name of the method used by the client. | com.myservice.EchoService/catchAll; com.myservice.EchoService/unknownMethod; InvalidMethod |
rpc.response.status_code | | Conditionally Required if available. | string | Status code of the RPC returned by the RPC server or generated by the client [4] | OK; DEADLINE_EXCEEDED; -32602 |
server.address | | Conditionally Required If available. | string | RPC server host name. [5] | example.com; 10.1.2.80; /tmp/my.sock |
server.port | | Conditionally Required [6] | int | Server port number. [7] | 80; 8080; 443 |
network.peer.address | | Recommended | string | Peer address of the network connection - IP address or Unix domain socket name. | 10.1.2.80; /tmp/my.sock |
network.peer.port | | Recommended If network.peer.address is set. | int | Peer port number of the network connection. | 65123 |
network.protocol.name | | Recommended | string | OSI application layer or non-OSI equivalent. [8] | http |
network.protocol.version | | Recommended | string | The actual version of the protocol used for network communication. [9] | 1.1; 2 |
network.transport | | Recommended | string | OSI transport layer or inter-process communication method. [10] | tcp; udp |
[1] rpc.system.name: The client and server RPC systems may differ for the same RPC interaction. For example, a client may use Apache Dubbo or Connect RPC to communicate with a server that uses gRPC since both protocols provide compatibility with gRPC.
[2] error.type: If the RPC fails with an error before status code is returned,
error.type SHOULD be set to the exception type (its fully-qualified class name, if applicable)
or a component-specific, low cardinality error identifier.
If a response status code is returned and status indicates an error,
error.type SHOULD be set to that status code. Check system-specific conventions
for the details on which values of rpc.response.status_code are considered errors.
The error.type value SHOULD be predictable and SHOULD have low cardinality.
Instrumentations SHOULD document the list of errors they report.
If the request has completed successfully, instrumentations SHOULD NOT set
error.type.
[3] rpc.method: The method name MAY have unbounded cardinality in edge or error cases.
Some RPC frameworks or libraries provide a fixed set of recognized methods for client stubs and server implementations. Instrumentations for such frameworks MUST set this attribute to the original method name only when the method is recognized by the framework or library.
When the method is not recognized, for example, when the server receives
a request for a method that is not predefined on the server, or when
instrumentation is not able to reliably detect if the method is predefined,
the attribute MUST be set to _OTHER. In such cases, tracing
instrumentations MUST also set rpc.method_original attribute to
the original method value.
If the RPC instrumentation could end up converting valid RPC methods to
_OTHER, then it SHOULD provide a way to configure the list of recognized
RPC methods.
The rpc.method can be different from the name of any implementing
method/function.
The code.function.name attribute may be used to record the fully-qualified
method actually executing the call on the server side, or the
RPC client stub method on the client side.
[4] rpc.response.status_code: Usually it represents an error code, but may also represent partial success, warning, or differentiate between various types of successful outcomes.
Semantic conventions for individual RPC frameworks SHOULD document what rpc.response.status_code means in the context of that system and which values are considered to represent errors.
[5] server.address: May contain server IP address, DNS name, or local socket name. When host component is an IP address, instrumentations SHOULD NOT do a reverse proxy lookup to obtain DNS name and SHOULD set server.address to the IP address provided in the host component.
[6] server.port: if server.address is set and if the port is supported by the network transport used for communication.
[7] server.port: When observed from the client side, and when communicating through an intermediary, server.port SHOULD represent the server port behind any intermediaries, for example proxies, if it’s available.
[8] network.protocol.name: The value SHOULD be normalized to lowercase.
[9] network.protocol.version: If protocol version is subject to negotiation (for example using ALPN), this attribute SHOULD be set to the negotiated version. If the actual protocol version is not known, this attribute SHOULD NOT be set.
[10] network.transport: The value SHOULD be normalized to lowercase.
Consider always setting the transport when setting a port number, since a port number is ambiguous without knowing the transport. For example different processes could be listening on TCP port 12345 and UDP port 12345.
The following attributes can be important for making sampling decisions and SHOULD be provided at span creation time (if provided at all):
error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
_OTHER | A fallback error value to be used when the instrumentation doesn’t define a custom value. | |
network.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
pipe | Named or anonymous pipe. | |
quic | QUIC | |
tcp | TCP | |
udp | UDP | |
unix | Unix domain socket | |
rpc.system.name has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
connectrpc | Connect RPC | |
dubbo | Apache Dubbo | |
grpc | gRPC | |
jsonrpc | JSON-RPC | |
RPC server span
Status:
This span represents an incoming Remote Procedure Call (RPC).
RPC server spans SHOULD cover the entire server-side lifecycle of an RPC, starting when the request is received and ending when the response is sent or the RPC is terminated due to an error or cancellation.
For streaming RPCs, the span SHOULD cover the full lifetime of the request and/or response streams until they are closed or terminated.
Span name: refer to the Span Name section.
Span kind MUST be SERVER.
Span status Refer to the Recording Errors document for details on how to record span status.
Attributes:
| Key | Stability | Requirement Level | Value Type | Description | Example Values |
|---|---|---|---|---|---|
rpc.system.name | | Required | string | The Remote Procedure Call (RPC) system. [1] | grpc; dubbo; connectrpc |
error.type | | Conditionally Required If and only if the operation failed. | string | Describes a class of error the operation ended with. [2] | DEADLINE_EXCEEDED; java.net.UnknownHostException; -32602 |
rpc.method | | Conditionally Required if available. | string | The fully-qualified logical name of the method from the RPC interface perspective. [3] | com.example.ExampleService/exampleMethod; EchoService/Echo; _OTHER |
rpc.method_original | | Conditionally Required If and only if it’s different than rpc.method. | string | The original name of the method used by the client. | com.myservice.EchoService/catchAll; com.myservice.EchoService/unknownMethod; InvalidMethod |
rpc.response.status_code | | Conditionally Required if available. | string | Status code of the RPC returned by the RPC server or generated by the client [4] | OK; DEADLINE_EXCEEDED; -32602 |
server.address | | Conditionally Required If available. | string | RPC server host name. [5] | example.com; 10.1.2.80; /tmp/my.sock |
server.port | | Conditionally Required [6] | int | Server port number. [7] | 80; 8080; 443 |
client.address | | Recommended | string | Client address - domain name if available without reverse DNS lookup; otherwise, IP address or Unix domain socket name. [8] | client.example.com; 10.1.2.80; /tmp/my.sock |
client.port | | Recommended | int | Client port number. [9] | 65123 |
network.peer.address | | Recommended | string | Peer address of the network connection - IP address or Unix domain socket name. | 10.1.2.80; /tmp/my.sock |
network.peer.port | | Recommended If network.peer.address is set. | int | Peer port number of the network connection. | 65123 |
network.protocol.name | | Recommended | string | OSI application layer or non-OSI equivalent. [10] | http |
network.protocol.version | | Recommended | string | The actual version of the protocol used for network communication. [11] | 1.1; 2 |
network.transport | | Recommended | string | OSI transport layer or inter-process communication method. [12] | tcp; udp |
[1] rpc.system.name: The client and server RPC systems may differ for the same RPC interaction. For example, a client may use Apache Dubbo or Connect RPC to communicate with a server that uses gRPC since both protocols provide compatibility with gRPC.
[2] error.type: If the RPC fails with an error before status code is returned,
error.type SHOULD be set to the exception type (its fully-qualified class name, if applicable)
or a component-specific, low cardinality error identifier.
If a response status code is returned and status indicates an error,
error.type SHOULD be set to that status code. Check system-specific conventions
for the details on which values of rpc.response.status_code are considered errors.
The error.type value SHOULD be predictable and SHOULD have low cardinality.
Instrumentations SHOULD document the list of errors they report.
If the request has completed successfully, instrumentations SHOULD NOT set
error.type.
[3] rpc.method: The method name MAY have unbounded cardinality in edge or error cases.
Some RPC frameworks or libraries provide a fixed set of recognized methods for client stubs and server implementations. Instrumentations for such frameworks MUST set this attribute to the original method name only when the method is recognized by the framework or library.
When the method is not recognized, for example, when the server receives
a request for a method that is not predefined on the server, or when
instrumentation is not able to reliably detect if the method is predefined,
the attribute MUST be set to _OTHER. In such cases, tracing
instrumentations MUST also set rpc.method_original attribute to
the original method value.
If the RPC instrumentation could end up converting valid RPC methods to
_OTHER, then it SHOULD provide a way to configure the list of recognized
RPC methods.
The rpc.method can be different from the name of any implementing
method/function.
The code.function.name attribute may be used to record the fully-qualified
method actually executing the call on the server side, or the
RPC client stub method on the client side.
[4] rpc.response.status_code: Usually it represents an error code, but may also represent partial success, warning, or differentiate between various types of successful outcomes.
Semantic conventions for individual RPC frameworks SHOULD document what rpc.response.status_code means in the context of that system and which values are considered to represent errors.
[5] server.address: May contain server IP address, DNS name, or local socket name. When host component is an IP address, instrumentations SHOULD NOT do a reverse proxy lookup to obtain DNS name and SHOULD set server.address to the IP address provided in the host component.
[6] server.port: if server.address is set and if the port is supported by the network transport used for communication.
[7] server.port: When observed from the client side, and when communicating through an intermediary, server.port SHOULD represent the server port behind any intermediaries, for example proxies, if it’s available.
[8] client.address: When observed from the server side, and when communicating through an intermediary, client.address SHOULD represent the client address behind any intermediaries, for example proxies, if it’s available.
[9] client.port: When observed from the server side, and when communicating through an intermediary, client.port SHOULD represent the client port behind any intermediaries, for example proxies, if it’s available.
[10] network.protocol.name: The value SHOULD be normalized to lowercase.
[11] network.protocol.version: If protocol version is subject to negotiation (for example using ALPN), this attribute SHOULD be set to the negotiated version. If the actual protocol version is not known, this attribute SHOULD NOT be set.
[12] network.transport: The value SHOULD be normalized to lowercase.
Consider always setting the transport when setting a port number, since a port number is ambiguous without knowing the transport. For example different processes could be listening on TCP port 12345 and UDP port 12345.
The following attributes can be important for making sampling decisions and SHOULD be provided at span creation time (if provided at all):
error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
_OTHER | A fallback error value to be used when the instrumentation doesn’t define a custom value. | |
network.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
pipe | Named or anonymous pipe. | |
quic | QUIC | |
tcp | TCP | |
udp | UDP | |
unix | Unix domain socket | |
rpc.system.name has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
connectrpc | Connect RPC | |
dubbo | Apache Dubbo | |
grpc | gRPC | |
jsonrpc | JSON-RPC | |
Distinction from HTTP spans
HTTP calls can generally be represented using just HTTP spans.
If they address a particular remote service and method known to the caller, i.e., when it is a remote procedure call transported over HTTP, the rpc.* attributes might be added additionally on that span, or in a separate RPC span that is a parent of the transporting HTTP call.
Note that method in this context is about the called remote procedure and not the HTTP verb (GET, POST, etc.).
Feedback
Was this page helpful?
Thank you. Your feedback is appreciated!
Please let us know how we can improve this page. Your feedback is appreciated!