Allow disk access from multiple instances and services.
in progress
M
Murphy Randle
We've been moving our main datastore off of Couchbase for a few years now, but I've still got to run it. Right now I have it running on four or five very large instances on EC2. Each instance has its own 1TB or so of disk space, and the database runs in cluster mode, with each of the nodes doing service discover to find each other. It'd be very nice to run this on Render. I could make a distinct service per node and bootstrap the cluster by hand, but it might be nice to be able to just configure the number of instances, and have each instance get its own disk space automatically.
Log In
T
Thomas Wright
Hey folks! Thomas from Render here. We just released an Early Access feature that will help with this - the ability to scale services with a disk to multiple instances, each with their own disk. You can read more about this feature here: https://docs.render.com/disks#scaling-disk-backed-services-early-access
Anurag Goel
in progress
C
Christopher Moody
I'm also interested in multiple services accessing a shared read-only disk
L
Lorenzo Marzullo
Any news on this?
A
Andrew Schmidt
This is a pretty big blocker for me. Our services will vary greatly in their usage by the hour/day. It would be great to have persistant storage and services that could scale out and in.
S
Sven Schwyn
This turns out to be a pretty hefty blocker for us. We have one or two features, concurrency handled, which require a shared disk. Hope this sees the day of light soonish!
Anurag Goel
Merged in a post:
Manual and Auto scaling for services with SSD Storage
E
Erik Bjerke
I don't see any reason why not to allow at least manual scaling of a service when it has an SSD Storage attached. A support agent of yours said that doing that would require to restart the service and I don't see why is that even considered a problem instead of not being able to scale when necessary.
Anurag Goel
We can't scale services with disks because only a single instance can write to/read from them right now. We plan to allow multiple readers/writers for disks, which would enable scaling right away.
E
Erik Bjerke
Anurag Goel: great!!, is there an ETA?
Anurag Goel
Erik Bjerke: No ETA yet. We'll update this issue once we're closer to working on it.
b
brian
Anurag Goel: this would be huge! Lack of zero downtime deploys on services with disks is pretty difficult
b
brian
This would be immensely helpful for scaling WordPress sites where it's difficult to make them truly stateless. Media, log files, CSV import/export, etc. get written to disk. I've done a lot of work to get a reproducible build but the /uploads directory needs to be persistent.
For my use case sharing across instances would be a game changer.
Murphy Randle have you found any alternatives? There are plugins to send media uploads to S3 but the other files I mentioned make media-only plugins an incomplete solution. I'm considering turning my /uploads directory (ignoring media) into a git repo and committing, fetching/pulling in a cron job to synchronize across instances.
There are also Docker storage drivers to support NFS volumes but I don't think those are usable with Render.
Anurag Goel any updates on whether this is being worked on or an ETA?
M
Murphy Randle
brian: I just ended up changing my application code to use S3 for media persistence. Not ideal for what I wanted, but it moved me forward.
Anurag Goel
brian: we aren't working on this at the moment given other requests. I see a very clear need for this feature, but our bandwidth and priorities for 2021 make it unlikely we'll roll it out this year. Sorry about that.
N
Noah Jahn
Would this also cover the case where auto-scaled instances share the same disk data in each instance?
Load More
→