Forums

Problem Flask app running pymongo on PythonAnywhere fails in certain circumstances giving error similar to:

No replica set members match selector Primary(), Timeout: 30s, Topology Description: topology_type: ReplicaSetNoPrimary, servers: [ 27017) server_type: RSSecondary, rtt: 0.026010908652096987>, (xx-xxxxxxx-shard-00-01.xxxxxxx.mongodb.net, 27017) server_type: Unknown, rtt: None, error=AutoReconnect(xx-xxxxxxx-shard-00-01.xxxxxxx.mongodb.net:27017: connection pool paused)>, 27017) server_type: RSSecondary, rtt: 0.026285539381206036>]>

Root Cause Importing an established/connected pymongo.MongoClient from a module outside of flask_app.py

Resolution

Create MongoClient instance inside flask_app.py itself using PythonAnywhere recommended settings e.g.

client = MongoClient(
    MONGO_URI,
    connectTimeoutMS=30000,
    socketTimeoutMS=None,
    connect=False,
    maxPoolsize=1,
)

Then move any imported functions/classes/methods that use MongoDB inside flask_app.py (or write them in other modules in such a way that they take 'client' as an injected argument)

Just to check, is that what solved your problem?

A bit of background: when you start a website on PythonAnywhere, we initially spin up a single process, which imports all of your code. That will run all code that it outside functions. So if you define a MongoDB client variable outside your functions, it will be created at that point. If you don't specify connect=False, it will create a connection to the MongoDB server.

As the next step, we "fork" that single process that is running your code into multiple worker processes -- this is so that your code can process multiple requests in parallel. This can cause problems if you don't have connect=False set, because all of those subprocesses will share the same connection to the Mongo server -- so if they try to send stuff at the same time, it will be mixed together, and responses will come back to processes for queries they never made.

So you need to put in the connect=False so that each process only tries to create a connection after the fork has happened.

One gotcha: if you use the client to make queries, even if you set connect=False, in code that is run before the fork, then it will create a connection anyway, and that connection may be re-used by the child processes -- leading to the same errors as if you hadn't set the connect=False parameter.

Thanks for the extra info Giles - what you mention at the end of your reply was what was happening in my case... I'd set connect=False just like you describe, but I'd then used the connection to pre-load some results and then imported those results and the client object.

Thanks for confirming that!