Skip to content

Add Microsoft Web IQ as a search and scraper provider#261

Open
mainhanu wants to merge 4 commits into
danny-avila:mainfrom
mainhanu:main
Open

Add Microsoft Web IQ as a search and scraper provider#261
mainhanu wants to merge 4 commits into
danny-avila:mainfrom
mainhanu:main

Conversation

@mainhanu

@mainhanu mainhanu commented Jun 23, 2026

Copy link
Copy Markdown

Summary

Adds microsoftWebIQ as a new option for both the search provider and the scraper provider in the search tool, alongside the existing Serper / SearXNG / Tavily / Firecrawl integrations.

core logic is src/tools/search/microsoft-scraper.ts & src/tools/search/microsoft-search.ts(~ 500 lines), most of the rest code is for ut & local test script

What's included

  • Search API (microsoft-search.ts) — wraps the Web IQ search endpoints:
    • POST /v3/search/web (with contentFormat: 'passage' for query-relevant snippets) → organic.
    • POST /v3/search/newstopStories + news, mirroring how Serper folds news into top stories.
    • POST /v3/search/videosvideos (title, link, snippet, thumbnail, duration, channel, source, date).
    • POST /v3/search/imagesimages (title, image url, dimensions, thumbnail, host page link, source/domain).
    • safeSearch is mapped to the API's off / strict enum for the videos and images endpoints.
  • Browse scraper (microsoft-scraper.ts) — implements BaseScraper over POST /v3/browse with liveCrawl: 'none' (indexed content only). Non-200 responses (202 crawl-in-progress, 404 not-indexed) are treated as scrape failures so the source is skipped rather than retried.
  • Wiring — registers the provider in createSearchAPI and the scraper selector in tool.ts, including config plumbing (microsoftWebIQApiKey, microsoftWebIQBaseUrl, microsoftWebIQSearchOptions, microsoftScraperOptions). Videos and images are supported, so the parallel type: 'videos' / type: 'images' sub-searches fan out and merge into the result.
  • Types — adds MicrosoftWebIQSearchOptions, MicrosoftScraperConfig, MicrosoftScrapeOptions, and MicrosoftBrowseResponse; extends the SearchProvider / ScraperProvider unions.
  • Tests (microsoft.test.ts) — covers missing API key, empty query, web→organic, news→topStories, videos→videos, images→images mapping, request failure, and scraper 200/202/404 handling (11 tests).
  • Smoke script (scripts/microsoft_search.ts, npm run search:microsoft) — exercises web, news, video, and image search, the Browse scraper, and the full search tool against the live API.

Configuration

Credentials are read from config or MICROSOFT_WEBIQ_API_KEY / MICROSOFT_WEBIQ_BASE_URL (defaults to https://api.microsoft.ai/v3).

Testing

  • npx jest src/tools/search/microsoft.test.ts — 11/11 passing
  • tsc --noEmit and ESLint clean
  • Local linked to the LibreChat app, demo videos below (will raise PR when agent is merged and published).

search

librechat.search.mov

scraper

librechat.loader.-.1080p.mov

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant