The project that I am currently working on has a huge data set of static lookup data. And, we have been using Redis to store this data since the beginning of the project. We figured, redis would be the fastest as the whole data is in memory. However, in our production use we have found redis to be the bottleneck.
This is not really redis’ fault as the data access pattern that we have involves a huge number of lookups more than 10K lookups per request. Also, since redis runs on a single core, it isn’t able to use all the cores on our server. Add the network costs and the serialization costs to it and things add up very quickly.
This led me to do some benchmarking of redis against ets with our actual production data and (un)surprisingly we found that ets beats Redis for simple key value data. So, if you are using redis as a key value store. Please do yourself a favor and use ets (If you are using Elixir or erlang).
Go ahead and try it out by tweaking the count of records or the parallelism.
We found that the ets to redis performance gap actually grows as the parallelism increases.
Checkout the repository for the benchmark data: https://github.com/minhajuddin/redis_vs_ets_showdown
You can also check the reports at:
Here is the gist of the benchmark:
ets_get_1000: does an ets lookup 1000 times
redis_get_1000: does a redis lookup 1000 times using
ets_get_multi: does an ets lookup 1000 times
redis_get_multi: does a single
HMGET Redis lookup
Operating System: Linux
Name ips average deviation median