The node monitoring of Crawlab is completed by Redis. The schematic diagram is shown below.
The work node will update the heartbeat information in Redis continuously. Using 'HSET nodes
The master node periodically obtains the heartbeat information of the work node in redis. If there is a work node whose time stamp is 60 seconds ago, the node is considered to be offline, the information of the node will be deleted in redis, and it will be set to "offline" in mongodb; if the time stamp is within the past 60 seconds, the information of the node will be retained, and it will be set to "online" in mongodb.