uso de memoria con muchas claves de membase vs redis

I recently ran a test with membase, incrementing 60 million keys, each key of size 20-30 bytes, the values are less than the value of an integer. This cluster was across 3 16 GB boxes, 15 GB dedicated to a single bucket (replication=1) in membase. The build is membase-server-community_x86_64_1.7.1.1 on 64-bit ubuntu lucid boxes.


Initially, 10 million keys resided on 3 GB of memory. (3mil keys / GB) @60 million keys resided on 45 GB of memory. (1.33mil keys / GB)

In comparison, redis handles 9-10 million keys / GB @ 60 million keys. This ratio of keys per GB is consistent regardless of the dataset size.


Membase does not seem to scale well when faced with key heavy datasets. Is there any tuning/configuration that could help Membase in this use case?


PS I migrated from redis to membase because the latter seemed to offer more reliability against cache failure. However, this degradation of performance with large datasets is a bit too painful.

preguntado el 08 de noviembre de 11 a las 18:11

Curious where you found Redis lacking in reliability or ability to recover from failure. Approaches like Master->Slave replication, using an Append Only File for persistence, and using transactions or pipelining should mitigate any realistic risks of data loss for most use cases. -

In case of the master failing, redis should rebalance its replication to other servers. The cluster should ideally fail because it doesn't have enough resources to handle the load across all the boxes, not because there happened to be an unlucky failure on both the slave and master boxes. -

In case people want more information, I've also posted a similar question here on forums: Doesn't seem to be a lot of activity on the thread though, not sure how good the support will be. For now I've switched back to redis. -

There's a big different between linear scale costs and and nonlinear scale costs. Membase has them, Redis does not. If you're building anything big, Redis looks like a toy (albeit a shiny one). -

@yhm it seems redis is a good toy for building big things, for example (basically, very big web site running on redis (300000 queries/s, huge dataset)). -

0 Respuestas

No es la respuesta que estás buscando? Examinar otras preguntas etiquetadas or haz tu propia pregunta.