Frequent Error - Transport Error (Status Unknown)

[piyush-bansal1989]

Hi Team

I'm writing to seek your assistance with a frequent error I'm encountering:

status: Unknown, message: "transport error", details: [], metadata: MetadataMap { headers: {} }

This error occurs frequently, and I'm unsure what might be missing to resolve it.
Could you please provide some insights on what this error message might indicate and any troubleshooting steps I can take to address it?

[khaf]

Where do you encounter this error? As an exception in the PHP client? Do you have the result code to that exception?
It seems to me that you are hitting a timeout error in part of the code chain, quite possibly in the GRPC layer, but without knowing more about your operating environment (the API you use, your policies and timeouts, the network quality and latency, etc) I cannot help you more.

[piyush-bansal1989]

Where do you encounter this error? As an exception in the PHP client? Do you have the result code to that exception? It seems to me that you are hitting a timeout error in part of the code chain, quite possibly in the GRPC layer, but without knowing more about your operating environment (the API you use, your policies and timeouts, the network quality and latency, etc) I cannot help you more.

Yes, as exception in php client, No result code throw in this case.
Operating in aws ec2 instance, where 3 nodes of cluster is setup, and connecting through one client right now.
Currently I'm setting the following policies at put and get

write policy
$wp = new WritePolicy(); $wp->setRecordExistsAction(RecordExistsAction::Update()); $wp->setMaxRetries(3); $wp->setSocketTimeout(5000); $wp->setSendKey(true);

read policy
$rp = new ReadPolicy(); $rp->setMaxRetries(3); $rp->setSocketTimeout(5000); $rp->setSendKey(true);

[piyush-bansal1989]

Where do you encounter this error? As an exception in the PHP client? Do you have the result code to that exception? It seems to me that you are hitting a timeout error in part of the code chain, quite possibly in the GRPC layer, but without knowing more about your operating environment (the API you use, your policies and timeouts, the network quality and latency, etc) I cannot help you more.

Yes, as exception in php client, No result code throw in this case. Operating in aws ec2 instance, where 3 nodes of cluster is setup, and connecting through one client right now. Currently I'm setting the following policies at put and get

write policy $wp = new WritePolicy(); $wp->setRecordExistsAction(RecordExistsAction::Update()); $wp->setMaxRetries(3); $wp->setSocketTimeout(5000); $wp->setSendKey(true);

read policy $rp = new ReadPolicy(); $rp->setMaxRetries(3); $rp->setSocketTimeout(5000); $rp->setSendKey(true);

@khaf : Can you please confirm about this? or will you suggest the configuration?

[khaf]

My colleague @vmsachin is working on this ticket. I'll let him respond.

[vmsachin]

@piyush-bansal1989 could you please tell me the exact setup of how you have setup the client and server.

[drZloy]

I think I've found the reason for this error. I started investigating from the gRPC side, as @khaf suggested, and the problem turned out to be that the Rust client was persisting gRPC connections. This caused issues with PHP-FPM, since the persisted connection was being reused across multiple requests handled by the same worker. At some point the connection would be dropped, leading to subsequent failures.

After removing the connection persistence, I no longer experienced these errors. I believe persisting connections to ACM is redundant, since ACM already manages the connection pool for the AE DB. So why persist the connections to ACM at all?

[khaf]

@drZloy How did you remove the gRPC connection persistence?

[drZloy]

@khaf , please consider this commit in my fork: drZloy/aerospike-php-client@07fc3b1