Bench errors out when workerTimeout > 60m about elb-metrics HOT 4 OPEN

Per chat with @aarthykc, this is because of too many metric points on CloudWatch for a time period > 60 mins. Curious if any of the following would work, in the order of good solutions:

🎉 Break down a time duration into chunks of 60 minutes, collate all procured data.
🍨 Display the metrics collected for the first 60 minutes, at the very least, along with the message that the process timed out.

from elb-metrics.

@arunasank, thank you for tracking this down ^_^.

Break down a time duration into chunks of 60 minutes, collate all procured data.

Do you mean making the request in batches ?
👍 Yes, this will probably be more accurate in terms telling the end user what the exact elb metrics are.

from elb-metrics.

It's my understanding that we implemented this because by default we ask for these metrics with a 60s interval -- so asking for many hours of data at 60s periods, you'd end up with a lot of datapoints to evaluate. We should have CloudWatch do the aggregation work for us instead.

The sustainable solution here is to know that we're getting a larger time range, and adjust the period so that we're aggregating by 10 min or an hour or something.

@arunasank @aarthykc does this make sense? Happy to help implement this if so.

from elb-metrics.

@aarthykc @emilymcafee Haven't really read the bench code, so please bear with me if this is should be obvious. If we aggregate CW metrics by 10 mins or an hour, will that mean:

• Benching with a timeout < 10m would take 10 (13?)m?
• Bench with a timeout < 10m fails?

from elb-metrics.