This is similar to a previous question, but my log data shows the opposite problem. I am seeing rare requests that take perhaps 3s or more, but typical response times are 100ms or less. This is the log entry for one of the problematic responses:
LOAD_BALANCER_IP - - [20/Feb/2019:13:36:12 +0000] "POST /DevicePost HTTP/1.1" 200 16 "-" "-" "client-ip-redacted" 3.052 0.002 .
Here is the log format:
log_format timed_main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" '
'$request_time $upstream_response_time $pipe';
The 0.002 matches what the Node.js logs are reporting from the upstream server, and 3.052 matches what the Elastic Load Balancer is reporting as the response time is recorded. This was set up by AWS Elastic Beanstalk as the (mostly) default configuration for a Node.js application. The only changes were for this logging entry, and manually setting the Content-Type header (the client is an embedded device and their HTTP library doesn't include this header, which creates problems for Node.js Express).
I think this indicates a delay in Node.js, but it is sporadic and I cannot tell if it is some kind of garbage collection, a caching problem (there should be no caching), or something else. Any help would be appreciated.
(I'm sorry I don't know how to format the quotes. I can't find any instructions on what the admin called "quoting conventions typically used on mailing lists" in the post "Please Read Before Posting in this Forum").