Crawlora Go SDK

Go client for the public Crawlora API. Use it to call Crawlora scraping, search, social, developer, marketplace, travel, media, maps, finance, prediction-market, brand, and usage endpoints with generated service groups, typed parameter structs, operation constants, and typed response aliases.

• Runtime: Go 1.22+
• Auth: x-api-key
• Default API base URL: https://api.crawlora.net/api/v1
• Reference: operations and recipes

Install

Install the module from Git:

go get github.com/Crawlora-org/crawlora-go-sdk@latest

For reproducible builds, pin a released tag:

go get github.com/Crawlora-org/crawlora-go-sdk@TAG

API Key

Create or sign in to your Crawlora account at crawlora.net, then create an API key in the dashboard.

read -r CRAWLORA_API_KEY
export CRAWLORA_API_KEY

First Request

package main

import (
	"context"
	"fmt"
	"os"

	crawlora "github.com/Crawlora-org/crawlora-go-sdk"
)

func main() {
	client := crawlora.NewClient(
		crawlora.WithAPIKey(os.Getenv("CRAWLORA_API_KEY")),
	)

	response, err := client.Bing.Search(context.Background(), crawlora.Params{
		"q":     "coffee shops",
		"count": 10,
	})
	if err != nil {
		panic(err)
	}

	fmt.Printf("%#v\n", response)
}

Endpoint groups are generated from the public API contract, so common calls are available as methods such as client.Bing.Search(...), client.YouTube.Transcript(...), and client.Google.MapSearch(...).

Typed Calls

Typed endpoint variants are generated for every operation:

response, err := client.Bing.SearchTyped(ctx, crawlora.BingSearchParams{
	Q:     "coffee shops",
	Count: crawlora.Int(10),
})

Optional scalar fields use pointer helpers such as crawlora.String(...), crawlora.Int(...), crawlora.Bool(...), and crawlora.Float64(...).

You can also call by operation id with generated constants and typed response decoding:

response, err := crawlora.RequestTyped[crawlora.BingSearchResponse](
	client,
	ctx,
	crawlora.OperationBingSearch,
	crawlora.Params{"q": "coffee shops"},
)

Configuration

client := crawlora.NewClient(
	crawlora.WithAPIKey(os.Getenv("CRAWLORA_API_KEY")),
	crawlora.WithBaseURL("https://api.crawlora.net/api/v1"),
	crawlora.WithRetries(2),
	crawlora.WithRetryDelay(250*time.Millisecond),
	crawlora.WithHeader("x-client", "my-app"),
)

Per-request options can override headers, timeout, and response mode. Header names are matched case-insensitively, so request headers can override default auth, user-agent, and content headers without duplicating variants such as x-api-key and X-API-KEY:

response, err := client.Bing.Search(
	ctx,
	crawlora.Params{"q": "coffee shops"},
	crawlora.WithRequestTimeout(10*time.Second),
	crawlora.WithRequestHeader("x-request-id", "search-001"),
)

Text Responses

Most endpoints return JSON. Response mode must be crawlora.ResponseAuto, crawlora.ResponseJSON, or crawlora.ResponseText. Endpoints that support alternate text output, such as YouTube transcripts, can opt into text mode:

transcript, err := client.YouTube.Transcript(
	ctx,
	crawlora.Params{
		"id":     "VIDEO_ID",
		"format": "text",
	},
	crawlora.WithResponseType(crawlora.ResponseText),
)

Errors

Failed API calls return *crawlora.Error:

var apiErr *crawlora.Error
if errors.As(err, &apiErr) {
	fmt.Println(apiErr.Status, apiErr.Code, apiErr.Body)
}

The error includes HTTP Status, optional API Code, parsed Body, RawBody, response Headers, and the underlying parser or transport error when available. Retryable responses honor positive Retry-After headers, capped at 30 seconds. Context cancellation and deadline errors are returned directly so callers can match them with errors.Is.

Classify failures with the ErrClient (4xx), ErrServer (5xx), and ErrNetwork (transport) sentinels, or the IsClientError/IsServerError/IsNetworkError methods on *crawlora.Error:

if errors.Is(err, crawlora.ErrServer) {
	// retry or alert
}

Pagination

Paginate walks page/offset endpoints, invoking your callback per page and stopping when a page returns no data. Return crawlora.ErrStopPagination to stop early:

err := client.Paginate(ctx, "ebay-seller-feedback", crawlora.Params{"seller": "acme"}, func(page any) error {
	// handle page
	return nil
})

Examples

Runnable examples live under examples/ and skip cleanly when required environment variables are missing:

go run ./examples/bing-search
go run ./examples/youtube-transcript

Set CRAWLORA_BASE_URL to point examples at a staging or local API.

Module Notes

Go consumers use this repository directly as a Go module:

go get github.com/Crawlora-org/crawlora-go-sdk@latest

Pin an explicit released tag for production applications and upgrade intentionally.