Skip to content

Allow multiple schemas for one shot #393

@ppinchuk

Description

@ppinchuk

It might be good to allow users to provide multiple schemas, which would set up multiple parsers for the one shot extraction.

Key considerations:

  • Auto generating queries, heuristic keywords, and keywork scores will become tricker. Might be able to get around this by only merging the "features" and "$descriptions" into a "main" schema and using that as the template
  • Allowing custom prompts for text collection and text extraction will have to support a per-schema approach. Likely the safest way to do this is to use a dict keyed by the schema filename (no extension?), but it's kind of ugly. Could also disable custom prompts for a multi-schema input but this seems unnecessarily restrictive. Maybe instead the schema input itself is a dict and then keys can match between those? No matter what, this is kind of a bookkeeping nightmare
  • The auto generated collection/text extraction prompts as well as the parsers should be on a 1-per-input schema basis, with the data getting concatenated at the very end like we do for wind/solar

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions