Deploy spider

Crawlab deploys spiders automatically. The master node synchronizes the spider files on this node to all online nodes every 60 seconds. Therefore, after the user uploads the spider, it only needs to wait up to 60 seconds to run the spider task on all nodes.

Please refer to principles - Chapter of spider deployment for the principles of spider deployment and synchronization.