ApifyScheduler

A Scrapy scheduler that uses the Apify Request Queue to manage requests.

This scheduler requires the asyncio Twisted reactor to be installed.

Index

Methods

• __init__
• enqueue_request
• has_pending_requests
• next_request
• open

Methods

__init__

• __init__(): None

• Create a new instance.

  ---

  Returns None

enqueue_request

• enqueue_request(request): bool

• Add a request to the scheduler.

  This could be called from either from a spider or a downloader middleware (e.g. redirect, retry, ...).

  ---

  Parameters

  • request: Request

    The request to add to the scheduler.

  Returns bool

  True if the request was successfully enqueued, False otherwise.

has_pending_requests

• has_pending_requests(): bool

• Check if the scheduler has any pending requests.

  ---

  Returns bool

  True if the scheduler has any pending requests, False otherwise.

next_request

• next_request(): Request | None

• Fetch the next request from the scheduler.

  ---

  Returns Request | None

  The next request, or None if there are no more requests.

open

• open(spider): None

• Open the scheduler.

  ---

  Parameters

  • spider: Spider

    The spider that the scheduler is associated with.

  Returns None