Request Blocking

October 27, 2018

The new Request Blocker 3.0 feature allows you to customize your Navigation Auto-Request Blocking in four different levels for faster and more automated scraping usability.

The Request Blocker 3.0 features can be accessed from the Web Console and the Agent Builder.  Follow the directions below for each:

 

From the Web Console:

    1. Open the Web Console.

    2. Click on Agents in the top navigation bar.

    3. Click on the agent you’d like to apply Request Blocker 3.0.

    4. Click on the Agent Settings .

    5. In the drop-down menu under Settings, click Harvesting.

    6. In the Harvesting Settings pop up window, click Advanced.

    7. Under Auto-Block – Automatically block navigation requests choose which level of Request Blocking you’d like to use for your specific Agent. Here, only Levels 1 & 2 are available to avoid scenarios where agents might be ruined by immediately setting them to Level 3 or 4. To access all 4 levels, follow the directions below from the Agent Builder.

    8. Click Save.

From the Agent Builder:

    1. Open the Agent Builder.

    2. Click Start New Agent or Open an Existing Agent.

    3. Under the File drop-down menu, click on Settings.

    4. In the Agent Builder Settings pop-up, under Request Blocking, select the Auto-Block level you’d like to apply.

    5. Click Save.

 

Who needs Request Blocker 3.0 features?

Anyone who needs to:

  • Harvest data from websites with heavy content requests
  • Run agents faster
  • Cut time from manually blocking requests
  • Block CSS content
  • Block event-driven content
  • And more

What is included in the Request Blocker 3.0 feature set?

Request Blocker 3.0 includes 4 different levels of Auto Request Blocking.

Level 1: Standard

Level 2: Third party

Level 3: Event driven

Level 4: One request

For example, the quickest way to get from point A to point B is in a straight line. The same principle applies to web data extraction. If you can just extract the data you seek, without loading all the unnecessary extras like advertisements, images, media files, analytics, CSS, etc. then it makes sense that your web scraping Agents will extract data at a much faster pace. The image below will help you visualize exactly what is included in the Request Blocker 3.0 and why it will help your Agents run faster.

Which Mozenda account types have the Request Blocker 3.0 features?

All Mozenda accounts have access to the new Request Blocker 3.0 feature which includes four different auto request blocking levels.

Need more information?

We are anxious to answer any questions you may have about our products and services.

100% Privacy. You are that important to us. Privacy Policy

X