Scale out and scale in automatically based on a metric such as CPU utilization rate. Scale out should add a specified number of instances, run build and deploy, and then automatically add it to the cluster. Scale in should remove the instance from the cluster and destroy it.
Our team's considering switching to Render but not having autoscaling (either vertical or horizontal) is a blocker for us. Is this planned for 2020 or can we expect it to arrive later?
@Daniel Meechan: horizontal autoscaling is definitely planned for this year; most likely this fall or sooner.
For the metric, we've had good results with autoscaling based on queue time in the load balancer that is in front of the application servers. This is the approach taken by the Rails Autoscale addon on Heroku (https://elements.heroku.com/addons/rails-autoscale). We found queue time to be a much better metric for autoscaling than response time, which is used by the default Heroku autoscaler.
This is the number one feature preventing me from migrating my production servers to Render. Without this ability, I will either have to overpay excessively for servers, or expect downtime during spikes in traffic.
@Omri Cohen: which metrics would you like to scale on?
@Anurag Goel: average CPU %, number of concurrent connections, memory. With sane defaults but customizable.
EG: ~10s moving average CPU, memory, or max connections above 85% -> spin up another instance.
@Kory Nunn: sounds good.
@Anurag Goel: Also something like requests per minute threshold would be cool. Let's say if server gets 300 RPM then scale up. If it goes down 150 RPM you scale down. Would be also cool to connect this with time. For instance if for 10 minutes requests are lower than 150 RPM then scale down.
I tested RailsAutoscale on Heroku with queue time which sounds cool but my app has spikes of traffic in short period of time (in a few seconds there are big hits to API 80-150 RPS). And queue time wasn't working good enough (most of the time servers ended up over provisioned) despite testing various queue time thresholds.
I guess autoscaling is just complex problem and a lot of depends on how app is working, what endpoints are slow or what's the traffic during the day. I can think about idea to solve it like the one mentioned above but I have no clue if this would be good solution until tested on real production app.
A range that you can scale to would also be great! E.g. never drop below 2 instances and never go above 10.