Manage rate-limiting internally
Please consider managing incoming requests (with a load balancer or reverse proxy) so the client applications do not need to manage rate-limiting.
Incoming requests above the 'simultaneous' threshold could be queued while resources are scaled/allocated, rather than being rejected with a 429 response.
This would simplify integration.
6
votes
Andy
shared this idea
we will be providing a batch api to perform this. It’s currently planned, but no ETA.