Running Multiple Instances of the Same Agent

Content Grabber can run as many agents concurrently as your computer can handle, but by default, you can only run a single instance of each agent at the same time. There are a few scenarios where it's important to be able to run multiple concurrent instances of the same agent, and Content Grabber allows you to configure an agent to support this.

Web Applications

Running multiple instances of the same agent is particularly useful when running an agent directly from a website where you can have many web users visiting the website at the same time. Each web user can start the agent in their own session and the user will only see agent progress status and extracted data belonging to his session.

Please see the topic Sessions in the Programming Interface chapter for more information.

Splitting Large Web Scraping Tasks

Large web scraping tasks can take a long time to complete, but you can decrease this time by splitting up the task and have multiple instances of an agent working on the task. You can run instances of then same agent on multiple computers to further optimize the process.

Please see the topic Sessions in the Improving Performance and Reliability chapter for more information.