Skip to content
This repository was archived by the owner on Nov 20, 2025. It is now read-only.
This repository was archived by the owner on Nov 20, 2025. It is now read-only.

Verify Redis implementation #77

Description

@anjackson

Having run a couple of heavy sets of seeds through the Redis Frontier, and after PAUSING the crawl, some queues are in qs:active and not in qs:scheduled - specifically:

0 | 1632996435109 | com,twitter,www,+1
1 | 1632996380612 | com,twitter,

and the timestamps are significantly older than those in qs:scheduled indicating that they've been forgotten. They are in qs:known so scanning that would bring it back.

  • Track down if this is a problem!
  • Determine how to properly evaluate the reliabilty of a whole replacement frontier!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions