Redis pub/sub adding additional channels mid subscription?

The python-redis Redis and ConnectionPool classes inherit from threading. Local and this is producing the "magical" effects you're seeing Summary : your main thread and worker threads self. _redis_sub clients end up using two different connections to the server, but only the main thread's connection has issued the SUBSCRIBE command Details : Since the main thread is creating the self.

_redis_sub that client ends up being placed into main's thread-local storage. Next I presume the main thread does a client. Subscribe(channel) call.

Now the main thread's client is subscribed on connection 1. Next you start the self. _sub_thread worker thread which ends up having its own self.

_redis_sub attribute set to a new instance of redis. Client which constructs a new connection pool and establishes a new connection to the redis server This new connection has not yet been subscribed to your channel, so listen() returns immediately. So with python-redis you cannot pass an established connection with outstanding subscriptions (or any other stateful commands) between threads Depending on how you plan to implement your app you may need to switch to using a different client, or come up with some other way to communicate subscription state to the worker threads, e.g. Send subscription commands through a queue One other issue is that python-redis uses blocking sockets, which prevents your listening thread from doing other work while waiting for messages, and it cannot signal it wishes to unsubscribe unless it does so immediately after receiving a message.

The python-redis Redis and ConnectionPool classes inherit from threading. Local, and this is producing the "magical" effects you're seeing. Summary: your main thread and worker threads' self.

_redis_sub clients end up using two different connections to the server, but only the main thread's connection has issued the SUBSCRIBE command. Details: Since the main thread is creating the self. _redis_sub, that client ends up being placed into main's thread-local storage.

Next I presume the main thread does a client. Subscribe(channel) call. Now the main thread's client is subscribed on connection 1.

Next you start the self. _sub_thread worker thread which ends up having its own self. _redis_sub attribute set to a new instance of redis.

Client which constructs a new connection pool and establishes a new connection to the redis server. This new connection has not yet been subscribed to your channel, so listen() returns immediately. So with python-redis you cannot pass an established connection with outstanding subscriptions (or any other stateful commands) between threads.

Depending on how you plan to implement your app you may need to switch to using a different client, or come up with some other way to communicate subscription state to the worker threads, e.g. Send subscription commands through a queue. One other issue is that python-redis uses blocking sockets, which prevents your listening thread from doing other work while waiting for messages, and it cannot signal it wishes to unsubscribe unless it does so immediately after receiving a message.

I think the last issue is the killer. I need the listening thread to be able to add and delete subscriptions at will. – Tristan Mar 14 at 22:25.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.