Sorry if somewhat off-topic, but might help with getting more users. Redis is actually good enough and easy to install for most projects, but it has downside that it's "memory-only" (needs to fit in ram), and if you're in a tight container/vm – you'd better not rely on it solely.
So, there's another project – called kvrocks https://github.com/apache/kvrocks , which is for people which don't need in-memory perf of redis but would like to use its protocol and be still very performant. However, its devs never packaged a release in deb and other formats.
So, if you were to implement something similar ^ which will implement the protocol, have ability to grow on disk, and be available in distro repos (ideally) – would definitely hold a niche of many little projects which benefit from redis-like thing I've done.
Redis is for high frequency cache, disk latencies would slow it too much. Maybe putting something that would normally be evicted to disk could be valid compromise.
another major drawbacks of Redis is that It is single threaded and sync. you need expensive high frequency CPU if you need more performance. something that could benefit from multi core system and if you would want to use disk cache async is a must. single threading also affects scripts running on Redis side, when they execute whole Redis is waiting and is unable to fulfil other requests.
another place for improvement would be proper easier to use HA, and maybe also auto scaling.
Most projects need to start with something and know that they will "be good". Many will hit RAM limitations earlier than kvrocks latencies or single-threaded CPU limitations (if they will – usually author doesn't need advice). Most would benefit from "moving fast" (in terms of development).
Redis-compatible protocol guarantees that you can think about the things you've mentioned later. If project is successful – you'll find money to pay for RAM and move to pure Redis. If it's CPU-bound – you'll find multi-threaded alternatives.
So this is a pretty nice niche IMO actually.
Why not use a managed Redis or a large memory instance and run Redis yourself then? They aren't that much more expensive if you need to move fast.
Yes, but why pay for it if `sudo apt install zedis` works? Many use-cases for redis growing out of memory would actually be ok if it stores old values on disk for you and keeps hot items in cache (and does compression).
It’s not feasible to do that in a single threaded system. The main drawback of disk persistence is that you need a log of memory to keep the file buffer open and pointers. If the key you’re looking for is not on disk, you’ll get awful results, the rdb format will give you worst then Postgres performance. You have to use another file format to optimize for a larger cache
The Redis HA story is horrendous. I wonder if Sentinel is intentionally just crap enough to get your application locked in but then force you to pay them for hosted option.
It regularly gets itself into messed up state and even with graceful shutdown for node updates the official client libraries throw errors for up to a minute.
fwiw redis has persistence https://redis.io/docs/latest/operate/oss_and_stack/managemen...
That just makes it durable, the entire dataset still has to fit in memory.
The format was made for taking snapshots, it’s terrible for lookup.
If you're in a tight VM and are running Redis inside it, as opposed to a separate VM / cluster, you need an embedded DB, not Redis.
You can use the ancient BerkeleyDB which is supported everywhere for last like 30 years. Or a more recent LevelDB. Or even SQLite which is plenty fast on simple queries.
You'll have to tolerate the high latency of safely writing tings to a block device, compared to RAM. If your load is mostly reads, it should be OK.
Redka may be interesting to you: https://github.com/nalgeon/redka
I didn’t know that, that’s pretty cool
Redis compatible projects with larger-than-memory support do exists, e.g. https://microsoft.github.io/garnet/docs/welcome/features#tie...
No apt repo, C#.. kinda worse than kvrocks in that sense
Why C# makes it worse?