How to use ServiceStack Redis in a web application to take advantage of pub / sub paradigm?

Actually Redis Pub/Sub handles this scenario quite well, as Redis is an async non-blocking server it can hold many connections cheaply and it scales well Salvatore (aka Mr Redis :) describes the O(1) time complexity of Publish and Subscribe operations : You can consider the work of subscribing/unsubscribing as a constant time operation, O(1) for both subscribing and unsubscribing (actually PSUBSCRIBE does more work than this if you are subscribed already to many patterns with the same client) About memory, it is similar or smaller than the one used by a key, so you should not have problems to subscribe to millions of channels even in a small server So Redis is more than capable and designed for this scenario, but the problem as Tom pointed out in order to maintain a persistent connection users will need long-running connections (aka http-push / long-poll) and each active user will take its own thread. Holding a thread isn't great for scalability and technologically you would be better off using a non-blocking http server like Manos de Mono or node. Js which are both async and non-blocking and can handle this scenario.

Note: WebSockets is more efficient for real-time notifications over HTTP, so ideally you would use that if the users browser supports it and fallback to regular HTTP if they don't ( or fallback to use Flash for WebSockets on the client ) So it's not the Redis or its Pub/Sub that doesn't scale here, it's the number of concurrent connections that a threaded HTTP server like IIS or Apache that is the limit, with that said you can still support a fair amount of concurrent users with IIS ( this post suggests 3000 ) and since IIS is the bottleneck and not Redis you can easily just add an extra IIS server into the mix and distribute the load.

Actually Redis Pub/Sub handles this scenario quite well, as Redis is an async non-blocking server it can hold many connections cheaply and it scales well. Salvatore (aka Mr Redis :) describes the O(1) time complexity of Publish and Subscribe operations: You can consider the work of subscribing/unsubscribing as a constant time operation, O(1) for both subscribing and unsubscribing (actually PSUBSCRIBE does more work than this if you are subscribed already to many patterns with the same client). ... About memory, it is similar or smaller than the one used by a key, so you should not have problems to subscribe to millions of channels even in a small server.So Redis is more than capable and designed for this scenario, but the problem as Tom pointed out in order to maintain a persistent connection users will need long-running connections (aka http-push / long-poll) and each active user will take its own thread.

Holding a thread isn't great for scalability and technologically you would be better off using a non-blocking http server like Manos de Mono or node. Js which are both async and non-blocking and can handle this scenario. Note: WebSockets is more efficient for real-time notifications over HTTP, so ideally you would use that if the users browser supports it and fallback to regular HTTP if they don't (or fallback to use Flash for WebSockets on the client).

So it's not the Redis or its Pub/Sub that doesn't scale here, it's the number of concurrent connections that a threaded HTTP server like IIS or Apache that is the limit, with that said you can still support a fair amount of concurrent users with IIS (this post suggests 3000) and since IIS is the bottleneck and not Redis you can easily just add an extra IIS server into the mix and distribute the load.

Thanks for your response mythz. It sounds not good using Pub/Sub in a web context (under IIS) :( May be, the solution would be to develop a windows service (or a "daemon") which would be as subscriber. In this way, publishers (web app) could send their notifications toward this service.

– Olivier DUVAL Jun 23 at 9:07.

Redis publish/subscribe is not designed for this scenario - it requires a persistent connection to redis, which you have if you are writing a worker process but not when you are working with stateless web requests. A publish/subscribe system that works for end users over http takes a little more work, but not too much - the simplest approach is to use a sorted set for each channel and record the time a user last got notifications. You could also do it with a list recording subscribers for each channel and write to the inbox list of each of those users whenever a notification is added.

With either of those methods a user can retrieve their new notifications very quickly. It will be a form of polling rather than true push notifications, but you aren't really going to get away from that due to the nature of http. Technically you could use redis pub/sub with long-running http connections, but if every user needs their own thread with active redis and http connections, scalability won't be very good.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.