Create Multiple Jobs for an Agent

October 31, 2016

A job is an instance of an agent running on the Mozenda harvesting servers. Multiple jobs can be run concurrently for a single agent. By running multiple jobs concurrently with differing parameters, agents can collect large amounts of data more quickly.

 

    1. Open the Web Console.

    2. Click the Agents tab and click to open the agent you want to create multiple jobs for.

    3. Click the Jobs tab.

    4. Click New job.

    5. In the Create Job dialog, enter a name, a description (optional), and parameters for the job (optional). If the agent has been built to accept parameters (e.g., page numbers on which to operate, index numbers identifying a subset of data, or search criteria used to return search results to be scraped), parameter values can be added to the job to ignore unwanted data during extraction and collect only the desired data. Using parameters in this way, you can divide the agent’s workload among multiple jobs running concurrently.

    6. Click Create Job.

    7. Repeat steps 4 – 6 to create another job. If the first job is still running when the second job begins to run, the agent status shown at the top of the dashboard (which would usually be “Ready” or “Running”) will be “Multiple Jobs.”

Need more information?

We are anxious to answer any questions you may have about our products and services.

100% Privacy. You are that important to us. Privacy Policy

X