diff --git a/.gitbook.yaml b/.gitbook.yaml index 10d4d0fa4..af21543a8 100644 --- a/.gitbook.yaml +++ b/.gitbook.yaml @@ -180,6 +180,12 @@ redirects: builder-cookbook/dapps/oracles: build/cookbook/README.md builder-cookbook/overview: build/cookbook/README.md builder-cookbook/table-of-contents: build/cookbook/README.md + builder-cookbook/filecoin-pin/dapp-demo: build/cookbook/filecoin-pin/dapp-demo.md + builder-cookbook/filecoin-pin/erc-8004-agent-registration: build/cookbook/filecoin-pin/erc-8004-agent-registration.md + builder-cookbook/filecoin-pin/filecoin-pin-cli: build/cookbook/filecoin-pin/filecoin-pin-cli.md + builder-cookbook/filecoin-pin/github-action: build/cookbook/filecoin-pin/github-action.md + builder-cookbook/filecoin-pin/faq: build/cookbook/filecoin-pin/faq.md + builder-cookbook/filecoin-pin: build/cookbook/filecoin-pin/README.md community: getting-started/community/forums-and-FIPs.md community/chat-and-discussion-forums: getting-started/community/forums-and-FIPs.md community/contribute/contribution-tutorial: getting-started/community/ways-to-contribute.md diff --git a/SUMMARY.md b/SUMMARY.md index 9776c2b11..b3d175935 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -157,6 +157,12 @@ * [Cookbook](build/cookbook/README.md) * [Store data](build/cookbook/store-data.md) * [Retrieve data](build/cookbook/retrieve-data.md) + * [Filecoin Pin](build/cookbook/filecoin-pin/README.md) + * [Filecoin Pin CLI](build/cookbook/filecoin-pin/filecoin-pin-cli.md) + * [Filecoin Pin GitHub Action](build/cookbook/filecoin-pin/github-action.md) + * [Filecoin Pin dApp Demo](build/cookbook/filecoin-pin/dapp-demo.md) + * [Filecoin Pin for ERC-8004 Agents](build/cookbook/filecoin-pin/erc-8004-agent-registration.md) + * [FAQ](build/cookbook/filecoin-pin/faq.md) ## Reference diff --git a/book.json b/book.json index 17cf63e08..5cbd0737f 100644 --- a/book.json +++ b/book.json @@ -2,5 +2,5 @@ "structure": { "readme": "WELCOME.md" }, - "plugins": ["filecoin", "-highlight"] + "plugins": ["filecoin", "-highlight", "-search", "-lunr"] } diff --git a/build/cookbook/README.md b/build/cookbook/README.md index a9df11934..b2dee46e6 100644 --- a/build/cookbook/README.md +++ b/build/cookbook/README.md @@ -6,5 +6,6 @@ Practical recipes for building on Filecoin. Each recipe focuses on a specific ta * [Store data](store-data.md) — recipes for programmable storage of public or private data on Filecoin * [Retrieve data](retrieve-data.md) — recipes for fetching stored data from the network +* [Filecoin Pin](filecoin-pin/README.md) — recipes for using Filecoin Pin to store and retrieve data on Filecoin Onchain Cloud [Was this page helpful?](https://airtable.com/apppq4inOe4gmSSlk/pagoZHC2i1iqgphgl/form?prefill_Page+URL=https://docs.filecoin.io/build/cookbook/overview) diff --git a/build/cookbook/filecoin-pin/README.md b/build/cookbook/filecoin-pin/README.md new file mode 100644 index 000000000..99583fd73 --- /dev/null +++ b/build/cookbook/filecoin-pin/README.md @@ -0,0 +1,40 @@ +--- +description: Pin IPFS content to Filecoin using familiar IPFS tools and workflows. +--- + +# Filecoin Pin + +## Status + +Filecoin Pin is currently **alpha software** running on the Filecoin Calibration testnet. As of 2025-10-15, it is undergoing active development and not yet recommended for production use. Please register for updates and GA announcement at [filecoin.cloud](https://filecoin.cloud/). + +## What is Filecoin Pin? + +Filecoin Pin is a fully decentralized persistence layer for IPFS content using the global network of Filecoin storage providers with cryptographic guarantees. + +When you use Filecoin Pin, your IPFS files gain: + +* **Verifiable persistence** - Storage providers must cryptographically prove daily that they continue to store and serve your data +* **Economic incentives** - You only pay when storage proofs are successfully delivered and verified onchain +* **Decentralized infrastructure** - Your data can be stored across a global network of independent storage providers +* **Sovereign data** - Choose your providers, audit storage proofs and payments onchain, with no dependency on a single company +* **Seamless IPFS integration** - Continue using standard [IPFS Mainnet](https://docs.ipfs.tech/concepts/glossary/#mainnet) tooling like Kubo, Helia, and IPFS HTTP Gateways while gaining Filecoin's persistence guarantees + +## Who is Filecoin Pin for? + +Filecoin Pin is designed for developers building on IPFS who need trustless, economically-incentivized persistence for their content. Whether you're building dApps, workflows, websites, AI agents, or other applications, Filecoin Pin provides the missing persistence layer for IPFS. + +## How to Get Started + +Get started using Filecoin Pin today with: + +1. [Filecoin Pin CLI](filecoin-pin-cli.md) - Upload new or existing IPFS files directly to Filecoin via the command line. Perfect for developers who want to integrate Filecoin storage into scripts, workflows, or local development environments. +2. [Filecoin Pin GitHub Actions](github-action.md) - Use GitHub Actions to automatically publish websites or build artifacts to IPFS and Filecoin as part of your CI/CD pipeline. Ideal for static websites, documentation sites, and automated deployment workflows. +3. [Filecoin Pin dApp Demo](dapp-demo.md) - Run or fork a simple demo dApp that demonstrates Filecoin Pin in a browser-based application. Great for understanding how to integrate Filecoin Pin into web applications. +4. [Filecoin Pin for ERC-8004 Agents](erc-8004-agent-registration.md) - Learn how to register a trustless autonomous agent on the ERC-8004 Identity Registry with verifiable persistent storage for agent metadata using Filecoin Pin. + +## Learn More + +* **[FAQ](faq.md)** - Common questions about Filecoin Pin +* **[Filecoin Pin GitHub Repository](https://github.com/filecoin-project/filecoin-pin)** - Source code and technical documentation +* **[Community and Support](https://github.com/filecoin-project/filecoin-pin?tab=readme-ov-file#community-and-support)** - Join the community for real-time developer support and updates. diff --git a/build/cookbook/filecoin-pin/assets/eip8004-base-explorer-nft.png b/build/cookbook/filecoin-pin/assets/eip8004-base-explorer-nft.png new file mode 100644 index 000000000..07f698e6c Binary files /dev/null and b/build/cookbook/filecoin-pin/assets/eip8004-base-explorer-nft.png differ diff --git a/build/cookbook/filecoin-pin/assets/pinapp-download.gif b/build/cookbook/filecoin-pin/assets/pinapp-download.gif new file mode 100644 index 000000000..7b6444754 Binary files /dev/null and b/build/cookbook/filecoin-pin/assets/pinapp-download.gif differ diff --git a/build/cookbook/filecoin-pin/assets/pinapp-publish.gif b/build/cookbook/filecoin-pin/assets/pinapp-publish.gif new file mode 100644 index 000000000..eb722fbf0 Binary files /dev/null and b/build/cookbook/filecoin-pin/assets/pinapp-publish.gif differ diff --git a/build/cookbook/filecoin-pin/assets/pinapp-upload.gif b/build/cookbook/filecoin-pin/assets/pinapp-upload.gif new file mode 100644 index 000000000..836ac54c0 Binary files /dev/null and b/build/cookbook/filecoin-pin/assets/pinapp-upload.gif differ diff --git a/build/cookbook/filecoin-pin/assets/pinapp-viewproof.gif b/build/cookbook/filecoin-pin/assets/pinapp-viewproof.gif new file mode 100644 index 000000000..be4db3cbd Binary files /dev/null and b/build/cookbook/filecoin-pin/assets/pinapp-viewproof.gif differ diff --git a/build/cookbook/filecoin-pin/dapp-demo.md b/build/cookbook/filecoin-pin/dapp-demo.md new file mode 100644 index 000000000..f08ae8c62 --- /dev/null +++ b/build/cookbook/filecoin-pin/dapp-demo.md @@ -0,0 +1,111 @@ +--- +description: See an example of Filecoin Pin working end to end within a web context. +--- + +# Filecoin Pin dApp Demo + +## What You'll Build + +In this walkthrough, you’ll build a simple drag-and-drop file uploader that: + +* Stores IPFS files directly on Filecoin with built-in payments, all in browser! +* Tracks real-time upload progress through each step +* Retrieves data easily from IPFS Mainnet and the underlying Filecoin Service Provider +* Verifies persistent storage with on-chain Filecoin proofs +* Multi-user support with session-based authentication +* Seamlessly integrates with React, TypeScript, and Vite + +## Walkthrough Recording + +{% embed url="" %} + +## Setup + +We will start building by [forking the *filecoin-pin-website demo repo](https://github.com/filecoin-project/filecoin-pin-website/fork).* Make sure you have **Node.js 18+** and **npm 9+** installed. The dapp works with Filecoin Calibration testnet. + +This will take ~10min. ⏲️ + +### Step 1: Fork, Clone, and Install Dependencies + +```bash +# Fork the repo using the command below +# or visit [https://github.com/filecoin-project/filecoin-pin-website/fork](https://github.com/filecoin-project/filecoin-pin-website/fork) +gh repo fork filecoin-project/filecoin-pin-website + +git clone https://github.com/YOUR-USERNAME/filecoin-pin-website.git +cd filecoin-pin-website +npm install +``` + +### **Step 2: Set Up Your Filecoin Wallet** + +A Filecoin wallet is needed to send transactions on Filecoin and pay for the Filecoin storage service. + +This demo dapp supports two authentication methods: + +* **Private Key:** easiest for local development and learning. +* **Session Key:** recommended for production deployments - allows multiple users to share one wallet safely. + +💡 The demo repo supports deployment with a shared session key, allowing multiple users to safely upload files using the same wallet. It has hardcoded DEFAULT_WALLET_ADDRESS and DEFAULT_SESSION_KEY that are ready to go, but please do NOT use it for production. You can override these defaults with env vars, see instructions [here](https://github.com/filecoin-project/filecoin-pin-website/blob/main/CONTRIBUTING.md#local-setup)! + +**2.1 Get test FIL and test USDFC** + +If you are using your own wallet, you need to get test FIL and test USDFC to pay for the Filecoin storage service and transactions. + +1. Create or use an existing Filecoin wallet on the ****Calibration testnet**** ([such as MetaMask](../../../networks-and-tools/assets/metamask-setup.md)). + +2. Visit the [Filecoin Calibration Faucet](https://faucet.calibnet.chainsafe-fil.io/funds.html) to get free test FIL (to pay for transaction gas). + +3. Visit the [Filecoin USDFC faucet](https://forest-explorer.chainsafe.dev/faucet/calibnet_usdfc]) to get test USDFC, which is a USD stable coin backed by FIL that can be used to pay for services. + +### **Step 3: Run Your dApp** + +Fire up your local development server: + +```bash + npm run dev +``` + +Visit `http://localhost:5173` and you should see your dApp running! That is all it takes to set up your app. Now, let’s upload a file to see the magic happen! + +## Store IPFS Files on Filecoin + +The magic happens in one key file [**`src/hooks/use-filecoin-upload.ts`**](https://github.com/filecoin-project/filecoin-pin-website/blob/main/src/hooks/use-filecoin-upload.ts). This is where your IPFS files get uploaded to Filecoin. + +### Step 1: Upload the data + +* **Prepare Service** - Validates wallet balance and gets the Filecoin Warm Storage service initialized. +* **Create CAR** - Converts your file to an IPFS CAR (Content Addressed aRchive) file. +* **Upload** - Sends the CAR file to a Filecoin Storage Provider (SP). + +![Uploading a file to Filecoin](./assets/pinapp-upload.gif) + +### Step 2: Announce CIDs and confirm the transaction + +The Filecoin SP: + +* indexes the IPFS CAR file and publishes all the contained CIDs to the IPFS network via IPNI. +* commits to the Filecoin network via onchain transactions to store the data. Once the transaction is confirmed, your data is paid to be persisted on Filecoin. + +![Publishing CIDs and confirming the transaction](./assets/pinapp-publish.gif) + +### Step 3: Download the data + +Your data is available from both the IPFS Mainnet network using standard traditional IPFS tooling and/or directly from Filecoin SPs. + +![Downloading the stored data](./assets/pinapp-download.gif) + +### **Step 4: Verify your data storage** + +Filecoin storage providers submit cryptographic proofs regularly onchain to prove that they are storing your data, and you can verify and see it for yourself [on the PDP Scan](https://pdp.vxb.ai/calibration). + +![Viewing storage proofs on PDP Scan](./assets/pinapp-viewproof.gif) + +That is it - you now have a dapp with a drag-and-drop interface to store IPFS Files on Filecoin! + +## Next Steps + +1. Check back on the [filecoin-pin-website repo](https://github.com/filecoin-project/filecoin-pin-website) - it will continue to be updated as new functionality is brought to filecoin-pin. +2. Feel free to report any issues with the dApp demo to +3. Check out the the [other example uses of filecoin-pin](../). +4. Ask questions or get help with filecoin-pin in the [supported communcation channels](https://github.com/filecoin-project/filecoin-pin?tab=readme-ov-file#support-info). diff --git a/build/cookbook/filecoin-pin/erc-8004-agent-registration.md b/build/cookbook/filecoin-pin/erc-8004-agent-registration.md new file mode 100644 index 000000000..07bc0dfed --- /dev/null +++ b/build/cookbook/filecoin-pin/erc-8004-agent-registration.md @@ -0,0 +1,765 @@ +--- +description: How to use the Filecoin Pin CLI with ERC-8004 autonomous agents +--- + +# Filecoin Pin for ERC-8004 Agents + +Learn how to register a trustless autonomous agent on the ERC-8004 Identity Registry with verifiable persistent storage using Filecoin Pin for the agent registration file. + +*** + +## Overview + +This tutorial walks you through registering an [ERC-8004](https://eips.ethereum.org/EIPS/eip-8004) compliant agent with cryptographically-verified persistent storage on Filecoin. You'll create an agent card (metadata describing your agent's capabilities), store it on Filecoin & IPFS using Filecoin Pin, and register it on-chain as an NFT on Base Sepolia. + +**What you'll learn:** + +* How to create an ERC-8004 compliant agent card +* How to use Filecoin Pin for persistent, verifiable storage +* How to register an agent on the ERC-8004 Identity Registry +* How to verify Filecoin storage proofs and on-chain registration + +**What you'll build:** +A GitHub Integration Agent that references GitHub's official MCP server, demonstrating how real-world services can be integrated with ERC-8004. + +*** + +## Example Code and Scripts + +For example code and helper scripts to help with using Filecoin Pin and agent registration, check out the quickstart repository: + +**GitHub Repository**: [FilOzone/FilecoinPin-for-ERC8004](https://github.com/FilOzone/FilecoinPin-for-ERC8004) + +*** + +## Why Filecoin Pin for Agent Storage? + +Agent cards need persistent storage with provable guarantees. Unlike generic IPFS pinning services that may stop hosting your data without notice, Filecoin Pin provides: + +* ✅ **Cryptographic proof** your data is stored (daily PDP proofs) +* ✅ **Ongoing verification** ensures storage persistence +* ✅ **Decentralized** storage across a global network +* ✅ **IPFS compatible** - works with existing tools and gateways +* ✅ **Crypto payments** - onchain payments +* ✅ **Limited time - sponsored storage coming soon** available for ERC-8004 builders + +*** + +## Prerequisites + +### Required Tools + +Before starting, you'll need: + +1. **Filecoin Pin CLI** - Follow the complete setup guide here: + * [Filecoin Pin CLI Tutorial](filecoin-pin-cli.md) + * This covers wallet creation, testnet tokens (tFIL and USDFC), and payment setup + +2. **Foundry** - Ethereum development toolkit for contract interactions + + ```bash + curl -L https://foundry.paradigm.xyz | bash + foundryup + ``` + +3. **jq** (optional but recommended) - JSON processor for viewing outputs + + ```bash + # macOS + brew install jq + + # Ubuntu/Debian + sudo apt-get install jq + ``` + +### Required Tokens + +You'll need testnet tokens on **two networks**: + +#### Filecoin Calibration Testnet + +* **tFIL** (testnet Filecoin) - For gas fees + * Request tFIL from [Filecoin Calibration Faucet](https://faucet.calibnet.chainsafe-fil.io/funds.html) + * Amount requested: 100 tFIL +* **USDFC** (Filecoin stablecoin) - For storage payments + * Request test USDFC from [Filecoin Calibnet USDFC Faucet](https://forest-explorer.chainsafe.dev/faucet/calibnet_usdfc) + * Or Mint at [USDFC website](https://stg.usdfc.net) (requires tFIL as collateral) + * Amount needed: ~5 USDFC + +#### Base Sepolia Testnet + +* **Sepolia ETH** - For NFT minting and registration + * Request test ETH on Base Sepolia on [Faucet](https://www.alchemy.com/faucets/base-sepolia) + * Amount needed: ~0.001 ETH + +> **NOTE!** The same Ethereum wallet works on both Filecoin Calibration and Base Sepolia. You only need one private key. + +### ERC-8004 Registry Address + +We'll be using the reference ERC-8004 Identity Registry deployed on the Base Sepolia testnet: + +```text +0x8004A818BFB912233c491871b3d84c89A494BD9e +``` + +*** + +## Step 1: Create Your Agent Card + +An agent card is a JSON file that describes your agent's capabilities, endpoints, and trust model according to the [ERC-8004 specification](https://eips.ethereum.org/EIPS/eip-8004). + +### Create the Agent Card JSON + +Create a file named `github-agent-card.json`: + +```bash +cat > github-agent-card.json << 'EOF' +{ + "type": "https://eips.ethereum.org/EIPS/eip-8004#registration-v1", + "name": "GitHub Integration Agent", + "description": "AI agent providing GitHub repository, issue, and pull request management capabilities through GitHub's official MCP server. Enables automated code review, issue triage, PR management, and repository analysis.", + "image": "https://github.githubassets.com/images/modules/logos_page/GitHub-Mark.png", + "endpoints": [ + { + "name": "MCP", + "endpoint": "https://api.githubcopilot.com/mcp/", + "version": "1.0.0", + "capabilities": { + "tools": [ + { + "name": "repository_management", + "description": "Browse code, search files, analyze commits across GitHub repositories" + }, + { + "name": "issue_management", + "description": "Create, update, search, and manage GitHub issues with AI assistance" + }, + { + "name": "pull_request_management", + "description": "Review PRs, manage approvals, merge conflicts, and code reviews" + } + ] + } + }, + { + "name": "agentWallet", + "endpoint": "eip155:84532:0x0000000000000000000000000000000000000000" + } + ], + "registrations": [], + "supportedTrust": [ + "reputation" + ] +} +EOF +``` + +### Validate the JSON + +Verify your agent card is valid JSON: + +```bash +jq . github-agent-card.json +``` + +You should see the formatted JSON output with syntax highlighting. + +### Understanding the Agent Card Structure + +Key fields in the agent card: + +* **`type`** - Links to the ERC-8004 specification version +* **`name`** - Human-readable name for your agent +* **`description`** - What the agent does +* **`image`** - Avatar or logo URL +* **`endpoints`** - Array of service endpoints: + * **MCP endpoint** - Points to GitHub's official MCP server + * **agentWallet** - The agent's wallet address (chain:chainId:address format) +* **`capabilities`** - Tools and functions the agent provides +* **`supportedTrust`** - Trust mechanisms (reputation, stake, etc.) + +> **💡 Note**: This example uses GitHub's real public MCP server at `https://api.githubcopilot.com/mcp/`. You can replace this with your own MCP server endpoint. + +*** + +## Step 2: Upload to Filecoin Pin + +Now we'll store the agent card on Filecoin with PDP proofs. + +### Setup Payment System + +If this is your first time using Filecoin Pin, set up the payment system: + +```bash +export PRIVATE_KEY="0x..." # Your wallet private key +filecoin-pin payments setup --auto +``` + +This configures your wallet to pay for storage automatically. +This may take a few minutes to complete. + +You'll see output similar to: + +```bash +filecoin-pin payments setup --auto +┌ Filecoin Onchain Cloud Payment Setup +│ +│ Running in auto mode... +│ +◇ ✓ Connected to calibration +│ +◇ ✓ Balance check complete +│ +│ Account: +│ Wallet: 0x44896a716F7b5Ed343C6962b3D56FaA5377Cd052 +│ Network: calibration +│ Balances: +│ FIL: 354.9996 tFIL +│ USDFC wallet: 199.6428 USDFC +│ USDFC deposited: 0.0000 USDFC +│ +◇ ✓ Deposited 1.0000 USDFC +│ +│ Transaction details: +│ Deposit: 0x3b9cb71e21c9df895c9a2f50df437e0dfa5cf00d20bff6cf6deea0c231b5b128 +│ +◇ ━━━ Configuration Summary ━━━ +│ +│ Network: calibration +│ Deposit: 1.0000 USDFC +│ Storage: ~372.4 GiB for 1 month +│ Status: Ready to upload +│ +└ Payment setup completed successfully +``` + +> **NOTE!** You only need to run this once per wallet. Subsequent uploads will use the existing payment configuration. + +### Upload Your Agent Card + +Upload the agent card to Filecoin: + +```bash +filecoin-pin add --auto-fund github-agent-card.json +``` + +The `--auto-fund` flag ensures your storage provider wallet has sufficient funds. + +You'll see output similar to: + +```bash +filecoin-pin add --auto-fund github-agent-card.json +┌ Filecoin Pin Add +│ +◇ ✓ File validated (1.3 KiB) +│ +◇ ✓ Connected to calibration +│ +◇ ✓ Minimum payment setup verified (~0.066 USDFC required) +│ +◇ ✓ File packed with root CID: bafybeihhal5hlbylkibniig6j72wdrm7lr4nf6z47natleh2jkyosrg7di +│ +◇ ✓ IPFS content loaded (1.5 KiB) +│ +◇ ✓ Funding requirements met +│ +◑ Creating storage context..Provider 0xB709A785c765d7d3F7d94dbA367DA6a611D7972b failed ping test: fetch failed +◇ ✓ Storage context ready +│ +│ Storage Context +│ +│ Data Set ID: undefined +│ Provider: pspsps-calibnet +│ +◇ ━━━ Add Complete ━━━ +│ +│ Network: calibration +│ +│ Add Details +│ File: github-agent-card.json +│ Size: 1.5 KiB +│ Root CID: bafybeihhal5hlbylkibniig6j72wdrm7lr4nf6z47natleh2jkyosrg7di +│ +│ Filecoin Storage +│ Piece CID: bafkzcibdricannieziik7jobrwqia4qfq6g7cwxfspsppv5aa76uev4u6ek7awz5 +│ Piece ID: 0 +│ Data Set ID: 11653 +│ +│ Storage Provider +│ Provider ID: 11 +│ Name: pspsps-calibnet +│ Direct Download URL: https://calibnet.pspsps.io/piece/bafkzcibdricannieziik7jobrwqia4qfq6g7cwxfspsppv5aa76uev4u6ek7awz5 +│ +└ Add completed successfully + +``` + +> **NOTE!** Data storage on the Calibration Testnet has a retention period of approximately 1 week. For production use and to ensure your data remains available, please switch to Filecoin mainnet. + +### Save Important Values + +Copy these values from the output - you'll need them later: + +* **Root CID** - The IPFS content identifier (e.g., `bafybeihhal5hlbylkibniig6j72wdrm7lr4nf6z47natleh2jkyosrg7di`). +* **Dataset ID** - For checking PDP proof status (e.g., `11653`) + +> **⚠️ IMPORTANT**: The Token URI for ERC-8004 registration must include the filename! Format it as: +> +> ```text +> ipfs:///github-agent-card.json +> ``` + +### Verify IPFS Retrieval + +Test that your agent card is accessible via IPFS (it may take a few minutes to propagate!): + +```bash +# Replace with your actual CID +curl -s "https://ipfs.io/ipfs//github-agent-card.json" | jq . +``` + +You should see your agent card JSON returned: + +```bash +curl -s "https://ipfs.io/ipfs/bafybeihhal5hlbylkibniig6j72wdrm7lr4nf6z47natleh2jkyosrg7di/github-agent-card.json" | jq . +{ + "type": "https://eips.ethereum.org/EIPS/eip-8004#registration-v1", + "name": "GitHub Integration Agent", + "description": "AI agent providing GitHub repository, issue, and pull request management capabilities through GitHub's official MCP server. Enables automated code review, issue triage, PR management, and repository analysis.", + "image": "https://github.githubassets.com/images/modules/logos_page/GitHub-Mark.png", + "endpoints": [ + { + "name": "MCP", + "endpoint": "https://api.githubcopilot.com/mcp/", + "version": "1.0.0", + "capabilities": { + "tools": [ + { + "name": "repository_management", + "description": "Browse code, search files, analyze commits across GitHub repositories" + }, + { + "name": "issue_management", + "description": "Create, update, search, and manage GitHub issues with AI assistance" + }, + { + "name": "pull_request_management", + "description": "Review PRs, manage approvals, merge conflicts, and code reviews" + } + ] + } + }, + { + "name": "agentWallet", + "endpoint": "eip155:84532:0x0000000000000000000000000000000000000000" + } + ], + "registrations": [], + "supportedTrust": [ + "reputation" + ] +} +``` + +*** + +## Step 3: Register on Base Sepolia + +Now we'll register the agent on-chain as an ERC-8004 NFT on Base Sepolia. + +### Set Environment Variables + +```bash +export PRIVATE_KEY="0x..." # Your wallet private key +export TOKEN_URI="ipfs:///github-agent-card.json" # From Step 2 +export IDENTITY_REGISTRY="0x8004A818BFB912233c491871b3d84c89A494BD9e" +export BASE_SEPOLIA_RPC="https://sepolia.base.org" +``` + +> **⚠️ SECURITY WARNING**: Never commit your private key to version control or share it publicly. + +### Check Your Balance + +Ensure you have sufficient Base Sepolia ETH: + +```bash +cast balance --rpc-url $BASE_SEPOLIA_RPC --ether +``` + +You should have at least 0.001 ETH for the registration transaction. + +### Register the Agent + +Send the registration transaction: + +```bash +cast send $IDENTITY_REGISTRY \ + "register(string)" \ + "$TOKEN_URI" \ + --rpc-url $BASE_SEPOLIA_RPC \ + --private-key $PRIVATE_KEY +``` + +You'll see output similar to: + +```bash +cast send $IDENTITY_REGISTRY \ + "register(string)" \ + "$TOKEN_URI" \ + --rpc-url $BASE_SEPOLIA_RPC \ + --private-key $PRIVATE_KEY + +blockHash 0x8699f29397c8b6b921d5e3f754220fa84ba049a0cfed875dc5deffcfd5f5dcb9 +blockNumber 37478106 +contractAddress +cumulativeGasUsed 6675540 +effectiveGasPrice 1200000 +from 0x44896a716F7b5Ed343C6962b3D56FaA5377Cd052 +gasUsed 200928 +logs [{"address":"0x8004a818bfb912233c491871b3d84c89a494bd9e","topics":["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef","0x0000000000000000000000000000000000000000000000000000000000000000","0x00000000000000000000000044896a716f7b5ed343c6962b3d56faa5377cd052","0x000000000000000000000000000000000000000000000000000000000000018e"],"data":"0x","blockHash":"0x8699f29397c8b6b921d5e3f754220fa84ba049a0cfed875dc5deffcfd5f5dcb9","blockNumber":"0x23bdeda","blockTimestamp":"0x698b1c94","transactionHash":"0x0edda2928ec45aaa4091a2fa2cc863f249e78b86931aed31e2c5365d3b99175c","transactionIndex":"0x13","logIndex":"0xe0","removed":false},{"address":"0x8004a818bfb912233c491871b3d84c89a494bd9e","topics":["0xf8e1a15aba9398e019f0b49df1a4fde98ee17ae345cb5f6b5e2c27f5033e8ce7"],"data":"0x000000000000000000000000000000000000000000000000000000000000018e","blockHash":"0x8699f29397c8b6b921d5e3f754220fa84ba049a0cfed875dc5deffcfd5f5dcb9","blockNumber":"0x23bdeda","blockTimestamp":"0x698b1c94","transactionHash":"0x0edda2928ec45aaa4091a2fa2cc863f249e78b86931aed31e2c5365d3b99175c","transactionIndex":"0x13","logIndex":"0xe1","removed":false},{"address":"0x8004a818bfb912233c491871b3d84c89a494bd9e","topics":["0xca52e62c367d81bb2e328eb795f7c7ba24afb478408a26c0e201d155c449bc4a","0x000000000000000000000000000000000000000000000000000000000000018e","0x00000000000000000000000044896a716f7b5ed343c6962b3d56faa5377cd052"],"data":"0x00000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000059697066733a2f2f626166796265696868616c35686c62796c6b69626e696967366a37327764726d376c72346e66367a34376e61746c6568326a6b796f7372673764692f6769746875622d6167656e742d636172642e6a736f6e00000000000000","blockHash":"0x8699f29397c8b6b921d5e3f754220fa84ba049a0cfed875dc5deffcfd5f5dcb9","blockNumber":"0x23bdeda","blockTimestamp":"0x698b1c94","transactionHash":"0x0edda2928ec45aaa4091a2fa2cc863f249e78b86931aed31e2c5365d3b99175c","transactionIndex":"0x13","logIndex":"0xe2","removed":false},{"address":"0x8004a818bfb912233c491871b3d84c89a494bd9e","topics":["0x2c149ed548c6d2993cd73efe187df6eccabe4538091b33adbd25fafdb8a1468b","0x000000000000000000000000000000000000000000000000000000000000018e","0x2ac6109326e720d1435c0db66f7e35eda7839f52b6f1f5520a60788e132b4e39"],"data":"0x00000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000b6167656e7457616c6c6574000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001444896a716f7b5ed343c6962b3d56faa5377cd052000000000000000000000000","blockHash":"0x8699f29397c8b6b921d5e3f754220fa84ba049a0cfed875dc5deffcfd5f5dcb9","blockNumber":"0x23bdeda","blockTimestamp":"0x698b1c94","transactionHash":"0x0edda2928ec45aaa4091a2fa2cc863f249e78b86931aed31e2c5365d3b99175c","transactionIndex":"0x13","logIndex":"0xe3","removed":false}] +logsBloom 0x00000000000000000000000000000000000000000000000000000000000000000000000080000000001000000000010200000000000000000000000000000000000000000000000000000008000080000000000000000000000000000000000000000000a20000000000000008000800000000000000000000000010000000000000000000100000000000000000000080000000000000000000000000000000000000000000000040000000000000002000000000000000010000000000008000000002000000002000000000000000000000000000000000800000000020000000081000010000200000000000000000000000000000000000000004000000 +root +status 1 (success) +transactionHash 0x0edda2928ec45aaa4091a2fa2cc863f249e78b86931aed31e2c5365d3b99175c +transactionIndex 19 +type 2 +blobGasPrice +blobGasUsed 46176 +to 0x8004A818BFB912233c491871b3d84c89A494BD9e +daFootprintGasScalar 312 +l1BaseFeeScalar 1101 +l1BlobBaseFee 63508363 +l1BlobBaseFeeScalar 659851 +l1Fee 8679909892 +l1GasPrice 928642664 +l1GasUsed 2383 + +``` + +A `status` of `1` means the transaction succeeded and your agent NFT was minted! + +### Get Your Agent ID + +Your agent ID is the **token ID** of the ERC-721 NFT that was minted when you registered. You can extract it from the transaction logs with `cast` and `jq` (install `jq` if needed, e.g. `brew install jq` on macOS): + +1. Set `TX_HASH` to the `transactionHash` from your `cast send` output (e.g. `0x0edda2928ec45aaa4091a2fa2cc863f249e78b86931aed31e2c5365d3b99175c`). +2. Get the receipt and read the **Transfer** event's `tokenId` (fourth topic): + +```bash +export TX_HASH="0x..." # from your cast send output + +# Extract agent ID (token ID) from the Transfer event in the receipt logs +AGENT_ID=$(cast receipt $TX_HASH --rpc-url $BASE_SEPOLIA_RPC --json \ + | jq -r '.logs[] | select(.topics[0] == "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef") | .topics[3]' \ + | head -1 \ + | xargs cast --to-dec) +echo "Agent ID: $AGENT_ID" +``` + +This filters logs for the ERC-721 **Transfer** event and converts the `tokenId` topic to decimal. Use `$AGENT_ID` in the [Verify Registration](#verify-registration) step below. + +**Alternative:** You can also look up the transaction on [BaseScan (Base Sepolia)](https://sepolia.basescan.org/) → **Logs** tab → **Transfer** or **Registered** event, and read the `tokenId` / `agentId` from the event (e.g. [example transaction](https://sepolia.basescan.org/tx/0x0edda2928ec45aaa4091a2fa2cc863f249e78b86931aed31e2c5365d3b99175c) shows token ID 398). + +### Verify Registration + +Confirm your agent is registered correctly using the agent ID from the previous step (use `$AGENT_ID` if you extracted it with `cast`, or the value you read from BaseScan): + +```bash +cast call $IDENTITY_REGISTRY \ + "tokenURI(uint256)" \ + $AGENT_ID \ + --rpc-url $BASE_SEPOLIA_RPC +``` + +You will get ABI-encoded output similar to: + +```bash +0x00000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000059697066733a2f2f626166796265696868616c35686c62796c6b69626e696967366a37327764726d376c72346e66367a34376e61746c6568326a6b796f7372673764692f6769746875622d6167656e742d636172642e6a736f6e00000000000000 +``` + +The output is ABI-encoded. Decode it: + +```bash +cast --abi-decode "f()(string)" +``` + +You should see your Token URI returned: `ipfs:///github-agent-card.json`: + +```bash +cast --abi-decode "f()(string)" 0x00000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000059697066733a2f2f626166796265696868616c35686c62796c6b69626e696967366a37327764726d376c72346e66367a34376e61746c6568326a6b796f7372673764692f6769746875622d6167656e742d636172642e6a736f6e00000000000000 +"ipfs://bafybeihhal5hlbylkibniig6j72wdrm7lr4nf6z47natleh2jkyosrg7di/github-agent-card.json" +``` + +### View on Block Explorer + +Visit your agent on the Base Sepolia block explorer: + +```text +https://sepolia.basescan.org/nft/0x8004a818bfb912233c491871b3d84c89a494bd9e/ +``` + +Replace `` with your agent's ID (e.g., `398`). + +
Base Sepolia block explorer, showing NFT minted
+ +*** + +## Step 4: Check On-chain Storage Proofs + +Finally, let's verify that your agent card is persistently stored with cryptographic proofs. + +### Check PDP Proof Status + +Use the Dataset ID from Step 2: + +```bash +filecoin-pin data-set show 11653 # Replace with your Dataset ID +``` + +You'll see output like: + +```bash +filecoin-pin data-set show 11653 +┌ Filecoin Onchain Cloud Data Set Details for #11653 +│ +◇ ━━━ Data Set ━━━ +│ +│ Network: calibration +│ Client address: 0x44896a716F7b5Ed343C6962b3D56FaA5377Cd052 +│ +│ #11653 +│ Status: live +│ CDN add-on: disabled +│ +│ Provider +│ ID: 11 +│ Address: 0x682467D59F5679cB0BF13115d4C94550b8218CF2 +│ Name: pspsps-calibnet +│ Description: herding cats +│ Service URL: https://calibnet.pspsps.io +│ Active: yes +│ Location: C=DE;ST=Bavaria;L=Nuremberg +│ +│ Metadata +│ source: "filecoin-pin" +│ withIPFSIndexing: "" +│ +│ Payment +│ PDP rail ID: 12908 +│ Payer: 0x44896a716F7b5Ed343C6962b3D56FaA5377Cd052 +│ Payee: 0x682467D59F5679cB0BF13115d4C94550b8218CF2 +│ +│ Pieces +│ Total pieces: 1 +│ Total size: 1.5 KiB +│ Unique PieceCIDs: 1 +│ Unique IPFS Root CIDs: 1 +│ +│ #0 (active) +│ PieceCID: bafkzcibdricannieziik7jobrwqia4qfq6g7cwxfspsppv5aa76uev4u6ek7awz5 +│ Size: 1.5 KiB +│ Metadata +│ ipfsRootCID: "bafybeihhal5hlbylkibniig6j72wdrm7lr4nf6z47natleh2jkyosrg7di" +│ +│ +└ Data set inspection complete + +``` + +> **💡 Note**: PDP proofs may take up to 24 hours to begin after initial upload. This is normal. + +### Summary + +✅ Your agent is now: + +* 🔒 **Persistently stored** on Filecoin with cryptographic PDP proofs +* 🌐 **Registered on-chain** as an ERC-8004 NFT on Base Sepolia +* 🔍 **Discoverable** via the Identity Registry by any third party +* ✅ **Verifiable** - anyone can check storage proofs and on-chain data +* 🚀 **Ready to use** by other agents and applications + +*** + +## Step 5: Deploy on Mainnet + +Ready to move to production? This step covers deploying your agent on Filecoin mainnet and Base mainnet. + +### Required Tokens (Mainnet) + +You'll need real tokens on **two networks**: + +#### Filecoin Mainnet + +* **FIL** - For gas fees on Filecoin +* **USDFC** - For storage payments + +Get FIL and USDFC via the [USDFC Bridge](https://app.usdfc.net/#/bridge) - bridge from any token on any network to FIL and USDFC on Filecoin mainnet. You can also use [Sushi](https://www.sushi.com/filecoin/swap?token0=NATIVE&token1=0x80b98d3aa09ffff255c3ba4a241111ff1262f045) to swap FIL for USDFC. + +#### Base Mainnet + +* **ETH** - For NFT minting and registration on Base + +### ERC-8004 Registry Address (Base Mainnet) + +```text +0x8004A169FB4a3325136EB29fA0ceB6D2e539a432 +``` + +View on explorer: [Base Mainnet Registry](https://basescan.org/address/0x8004A169FB4a3325136EB29fA0ceB6D2e539a432) + +### Upload to Filecoin Mainnet + +All Filecoin Pin CLI commands use the `--mainnet` flag for mainnet operations. + +#### Setup Payment System (Mainnet) + +```bash +export PRIVATE_KEY="0x..." # Your wallet private key +filecoin-pin payments setup --auto --mainnet +``` + +#### Upload Your Agent Card (Mainnet) + +```bash +filecoin-pin add --auto-fund --mainnet github-agent-card.json +``` + +Save the **Root CID** and **Dataset ID** from the output. + +### Register on Base Mainnet + +#### Set Environment Variables + +```bash +export PRIVATE_KEY="0x..." # Your wallet private key +export TOKEN_URI="ipfs:///github-agent-card.json" # From previous step +export IDENTITY_REGISTRY="0x8004A169FB4a3325136EB29fA0ceB6D2e539a432" +export BASE_MAINNET_RPC="https://mainnet.base.org" +``` + +#### Register the Agent + +```bash +cast send $IDENTITY_REGISTRY \ + "register(string)" \ + "$TOKEN_URI" \ + --rpc-url $BASE_MAINNET_RPC \ + --private-key $PRIVATE_KEY +``` + +#### Verify Registration + +```bash +export TX_HASH="0x..." # from your cast send output + +# Extract agent ID (token ID) from the Transfer event in the receipt logs +AGENT_ID=$(cast receipt $TX_HASH --rpc-url $BASE_MAINNET_RPC --json \ + | jq -r '.logs[] | select(.topics[0] == "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef") | .topics[3]' \ + | head -1 \ + | xargs cast --to-dec) +echo "Agent ID: $AGENT_ID" +``` + +### Check Mainnet Storage Proofs + +```bash +filecoin-pin data-set show --mainnet +``` + +*** + +## Troubleshooting + +### Issue: `filecoin-pin: command not found` + +**Solution**: Install the Filecoin Pin CLI: + +```bash +npm install -g filecoin-pin@latest +``` + +### Issue: `Insufficient USDFC` + +**Solution**: Request more test USDFC at [Filecoin Calibnet USDFC Faucet](https://forest-explorer.chainsafe.dev/faucet/calibnet_usdfc) + +### Issue: `Transaction reverted` on Base Sepolia + +**Solution**: Check your Base Sepolia ETH balance: + +```bash +cast balance --rpc-url https://sepolia.base.org --ether +``` + +Get more from the [faucet](https://www.alchemy.com/faucets/base-sepolia) if needed. + +### Issue: IPFS retrieval is slow or fails + +**Solution**: IPFS propagation can take a few minutes. Try different gateways: + +```bash +curl -s "https://ipfs.io/ipfs//github-agent-card.json" | jq . +curl -s "https://gateway.pinata.cloud/ipfs//github-agent-card.json" | jq . +curl -s "https://cloudflare-ipfs.com/ipfs//github-agent-card.json" | jq . +``` + +### Issue: PDP proofs not showing + +**Solution**: PDP proofs can take up to 24 hours to begin after upload. This is normal - your data is still stored, proofs just take time to generate. Check back later with: + +```bash +filecoin-pin data-set show +``` + +### Issue: Token URI doesn't include filename + +**Solution**: The Token URI must include the full path including filename. Correct format: + +```text +ipfs:///github-agent-card.json +``` + +If you registered with the wrong format, you'll need to register a new agent with the corrected Token URI. + +*** + +## What's Next? + +Now that your agent is registered with verifiable persistent storage, you can: + +### Build Your Own Agent + +1. **Create custom agent cards** for your services +2. **Deploy your own MCP server** and reference it in the agent card +3. **Register multiple agents** for different capabilities +4. **Update agent cards** by uploading new versions (CID changes) and updating on-chain + +### Explore ERC-8004 Features + +* **Reputation Registry** - Build reputation for your agents +* **Validation Registry** - Add validators to verify agent behavior +* **Multi-agent coordination** - Discover and compose multiple agents + +### Join the Community + +* **Filecoin Builders**: [on telegram](https://t.me/+Xj6_zTPfcUA4MGQ1); [on Slack](https://filecoinproject.slack.com/archives/CRK2LKYHW) +* **ERC-8004 Discussion**: [GitHub Discussions](https://github.com/ethereum/EIPs/issues/8004) +* **Filecoin Pin**: [Documentation](README.md) +* **Builder Channels**: Join ERC-8004 builder communities + +### Sponsored Storage for ERC-8004 Builders + +Coming soon, stay tuned! + +*** + +## Additional Resources + +* [**ERC-8004 Specification**](https://eips.ethereum.org/EIPS/eip-8004) +* [**Reference Implementation**](https://github.com/ChaosChain/trustless-agents-erc-ri) +* [**Filecoin Pin CLI Tutorial**](filecoin-pin-cli.md) +* [**Base Sepolia Explorer**](https://sepolia.basescan.org) +* [**GitHub MCP Server**](https://github.com/github/github-mcp-server) + +*** + +**Happy building!** 🚀 diff --git a/build/cookbook/filecoin-pin/faq.md b/build/cookbook/filecoin-pin/faq.md new file mode 100644 index 000000000..6e35efc7a --- /dev/null +++ b/build/cookbook/filecoin-pin/faq.md @@ -0,0 +1,114 @@ +# FAQ + +## What is Filecoin Pin? + +Filecoin Pin stores IPFS content on the Filecoin Network of decentralized Storage Providers. It enables developers to programmatically pay for storage and retrieval with Filecoin Pay. When SPs prove storage, they are paid from the developers' Filecoin Pay Account. + +*** + +### How can I use Filecoin Pin today? + +Two paths are available: + +* **Website:** Upload files in your browser. Uses a pre-funded test wallet. +* **CLI:** Upload files from your terminal. Fund storage from your own wallet. + +{% hint style="info" %} +Both run on Calibration testnet. They use tFIL and USDFC. Data has no persistence guarantees while on Calibnet. +{% endhint %} + +*** + +### What do I need to get started? + +* You can find different links related to Filecoin Pin here: [Filecoin Pin documentation](README.md) + +*** + +### How do payments and approvals work? + +* **Website:** The demo wallet handles payments. It has been prefunded with testnet USDFC and FIL. Users don't need to connect their own wallet. +* **CLI:** Your test wallet handles payments. You approve and deposit funds through Filecoin Pay. + +{% hint style="info" %} +Storage providers receive payment after cryptographically proving data possession. +{% endhint %} + +*** + +### How does auto-funding work? + +Use `--auto-fund` when uploading. The CLI calculates storage costs automatically. It deposits the right amount of USDFC to your payment rail. + +{% hint style="info" %} +No manual deposit calculations needed. The system handles it. +{% endhint %} + +*** + +### How long is my data stored? + +This runs on Calibration testnet only. Treat it as a demo. No duration guarantees exist for Website or CLI. + +Mainnet will offer persistence guarantees. Data persists while you maintain deposits. The CLI supports auto-funding for storage. + +*** + +### What is a Data Set? + +A Data Set groups your uploads together. Each upload becomes a "piece" within the Data Set. Multiple files you upload share the same payment rail. + +Check your Data Set with `filecoin-pin data-set `. + +*** + +### How do I retrieve my data? + +Three methods: + +1. **IPFS Gateways:** Use public gateways with your root CID: `https://gateway.example.com/ipfs/` +2. **Direct from Storage Provider:** Get the direct download URL from `filecoin-pin data-set ` +3. **IPFS Tools:** Use Kubo, Helia, IPFS Desktop with your root CID. + +*** + +### What is a piece CID vs root CID? + +**Root CID** (bafybei...) is your IPFS content identifier. Use this to retrieve your data. + +**Piece CID** (bafkzci...) is the Filecoin commitment. Storage Providers prove they store this piece. + +{% hint style="info" %} +Both are linked cryptographically on-chain. +{% endhint %} + +*** + +### How do I verify my data is actually stored? + +Two ways to verify: + +1. **CLI:** Run `filecoin-pin data-set ` to see on-chain verification. Check proof status and piece details. +2. **PDP Explorer:** Visit `https://pdp.vxb.ai/calibration/dataset/{datasetID}` to view proofs in your browser. + +{% hint style="info" %} +Both methods show CommP and proof state directly from blockchain state. +{% endhint %} + +*** + +### How do I access the code for the dApp and CLI? + +See the repos as reference implementations and to fork for my own project? + +* **Website**: [https://github.com/filecoin-project/filecoin-pin-website](https://github.com/filecoin-project/filecoin-pin-website) +* **CLI:** [https://github.com/filecoin-project/filecoin-pin](https://github.com/filecoin-project/filecoin-pin) + +*** + +## References + +* Filecoin Pin CLI Docs: [Filecoin Pin documentation](README.md) +* Filecoin Pin dApp Repo: [https://github.com/filecoin-project/filecoin-pin-website](https://github.com/filecoin-project/filecoin-pin-website) +* Synapse SDK: [https://github.com/FilOzone/synapse-sdk](https://github.com/FilOzone/synapse-sdk) +* USDFC documentation: [https://docs.secured.finance/usdfc-stablecoin/getting-started](https://docs.secured.finance/usdfc-stablecoin/getting-started) diff --git a/build/cookbook/filecoin-pin/filecoin-pin-cli.md b/build/cookbook/filecoin-pin/filecoin-pin-cli.md new file mode 100644 index 000000000..d348749ba --- /dev/null +++ b/build/cookbook/filecoin-pin/filecoin-pin-cli.md @@ -0,0 +1,329 @@ +--- +description: How to use the Filecoin Pin CLI to store and retrieve IPFS data on Filecoin +--- + +# Filecoin Pin CLI + +> ⚠️ **SECURITY WARNING**: This tutorial uses throwaway private keys for demo purposes. **NEVER commit private keys to repositories or expose them publicly**. Use environment variables and GitHub secrets for production. + +## Overview + +Filecoin Pin CLI is a command-line tool that allows you to store IPFS data on the Filecoin network with cryptographic proofs of storage. This guide walks you through the complete setup and usage process. + +### Setup Payments + +Configure permissions for automatic payment handling: + +
+ +### Upload Data + +Upload your file with automatic funding: + +
+ +### Retrieve over an IPFS Gateway + +Retrieve your data using the IPFS gateway: + +
+ +### Prove Storage + +Verify your data is stored with cryptographic proofs: + +
+ +### Who is this for + +1. Existing IPFS developers who want to use Filecoin to persist their data +2. Agent builders that want to store their agent cards and validation materials on Filecoin for cryptographic proof of storage + +## Getting started + +> NOTE! For demo purposes, this example uses a THROWAWAY PRIVATE KEY. **NEVER USE YOUR PRIVATE KEY IN A REPOSITORY OR EXPOSE IT**. The repo references using your private key LOCALLY as an ENV VARIABLE. When you create a GITHUB ACTION, use GITHUB SECRETS to store your private key. + +### Prerequisites + +#### Install Required Tools + +```bash +# Install Node.js 22+ (required for filecoin-pin) +nvm install 22 +nvm use 22 + +# Install filecoin-pin globally +npm install -g filecoin-pin@latest + +# Install GitHub CLI +brew install gh # macOS +# or +sudo apt install gh # Ubuntu/Debian + +# Install Foundry (for wallet operations) +curl -L https://foundry.paradigm.xyz | bash +# Then run this in a new terminal: +foundryup +``` + +#### Verify Installation + +```bash +filecoin-pin --version +# Expected: filecoin-pin v0.9.1 (or later) + +gh --version +# Expected: gh version 2.40.0 (or later) + +node --version +# Expected: v22.x.x or higher + +cast --version +# Expected: cast 0.2.0 (or later) +``` + +*** + +### 1. Wallet Setup + +> **Note**: The `filecoin-pin` CLI expects a `PRIVATE_KEY` for your Filecoin wallet to pay for storage service and transaction fees of using Filecoin warm storage service. + +So the first step is to acquire some test FIL and USDFC on the Filecoin calibration testnet into your ETH-compatible wallet. + +#### Generate a New Wallet + +If you do not have an ETH-compatible Filecoin wallet, you can generate one using Foundry: + +```bash +cast wallet new +``` + +Save the private key and wallet address. + +#### Get Testnet tokens + +* **tFIL** + + Request 100 tFIL from [ChainSafe calibration faucet](https://faucet.calibnet.chainsafe-fil.io/funds.html). +* **Test USDFC** + + Request test USDFC from [Filecoin Calibnet USDFC Faucet](https://forest-explorer.chainsafe.dev/faucet/calibnet_usdfc) + +*** + +### 2. Create Environment File + +> This PRIVATE\_KEY is for testing purposes only. Please **NEVER USE YOUR PRIVATE KEY IN A REPOSITORY OR EXPOSE IT**. + +Save your credentials **locally** for easy reuse: + +```bash +cat > ~/.filecoin-pin-env << 'EOF' +export PRIVATE_KEY="0x8eef...c414" +export WALLET_ADDRESS="0x5a0...B7B0B" +EOF + +# Load variables +source ~/.filecoin-pin-env + +# Verify +echo "Wallet: $WALLET_ADDRESS" +echo "Private Key: ${PRIVATE_KEY:0:10}..." +``` + +*** + +### 3. Using the Filecoin Pin CLI + +#### Setup Payments + +All commands in this section use the environment variables set in [Create Environment File](filecoin-pin-cli.md#id-2.-create-environment-file). If you're starting a new terminal session, reload them: + +```bash +source ~/.filecoin-pin-env +``` + +> **Note:** These commands work directly without running a server/daemon. The `PRIVATE_KEY` environment variable must be set. + +Configure payment approvals (permissions only - deposits handled automatically with `--auto-fund`): + +```bash +filecoin-pin payments setup --auto +``` + +> **What `--auto` does**: Configures WarmStorage contract permissions automatically. No deposit required at this step - use `--auto-fund` when uploading to handle deposits automatically. + +#### Upload Data + +Use the `--auto-fund` flag to automatically handle payment deposits (v0.7.0+). + +**Upload a file:** + +```bash +# Create test file +echo "Hello Filecoin from CLI!" > demo.txt + +# Upload to Filecoin +filecoin-pin add demo.txt --auto-fund +``` + +**Key values explained:** + +* **Root CID**: `bafybeibh422kjvgfmymx6nr7jandwngrown6ywomk4vplayl4de2x553t4` - IPFS CID for your data, which you can access from IPFS gateways, such as **ipfs.io**. +* **Piece CID**: `bafkzcibcfab4grpgq6e6rva4kfuxfcvibdzx3kn2jdw6q3zqgwt5cou7j6k4wfq` - Filecoin piece commitment (cryptographic proof) +* **Piece ID**: `0` - Reference within the data set +* **Data Set ID**: `325` - On-chain data set containing your upload +* **Transaction**: `0xc85e49d2ed745cc8c5d7115e7c45a1243ec25da7e73e224a744887783afea42b` - Blockchain confirmation hash +* **Direct Download URL**: Direct link to retrieve your data from the storage provider + +✅ **Your file is now stored on Filecoin with ongoing proof of possession!** + +1. Your file will be accessible via IPFS gateways, such as `https://ipfs.io/ipfs/` +2. It is also available to download from the Filecoin service provider using the **Direct Download URL**. + +#### Upload Directory + +You can also upload a directory - it will package multiple files into a single CAR: + +```bash +# Create test directory +mkdir my-data +echo "File 1" > my-data/file1.txt +echo "File 2" > my-data/file2.txt +echo "File 3" > my-data/file3.txt + +# Upload entire directory with auto-funding +filecoin-pin add my-data/ --auto-fund +``` + +**Key details:** + +* **Root CID**: `bafybeig27btater5fpt3l67gbme3sebqk3ynwdhlbrbuk3q7espiyplan4` - IPFS CID for the directory structure +* **Size**: 433.0 B - Includes all files plus directory metadata +* **Piece ID**: `1` - Second piece in the same Data Set ID 325 +* **Data Set ID**: `325` - Same data set as the single file upload (multiple pieces grouped together) + +> 💡 **Note**: Multiple uploads to the same payment configuration are grouped into the same Data Set, with each upload assigned a unique Piece ID. + +#### Prove Storage + +Filecoin-pin CLI also provide commands to check the proof of your storage in data set. First, you can list the dataset associate to your wallet. + +```bash +filecoin-pin data-set --ls + +//Expected Output +━━━ Data Sets ━━━ +│ +│ Address: 0x5a0...B7B0B +│ Network: calibration +│ +│ #325 • live • managed +│ Provider: infrafolio-calib (ID 4) +│ Pieces stored: 3 +│ ... +│ +└ Data set inspection complete +``` + +Then you can get detailed information about your data set (this queries the blockchain directly) that includes proofs: + +```bash +filecoin-pin data-set 325 +``` + +**Expected Output:** + +```text +Filecoin Onchain Cloud Data Sets + +━━━ Data Sets ━━━ + +Data Set #325 • live + Managed by Warm Storage: yes + CDN add-on: disabled + Pieces stored: 2 + Leaf count: 21 + Total size: 672.0 B + Client data set ID: 0 + PDP rail ID: 631 + CDN rail ID: none + Cache-miss rail ID: none + Payer: 0x5a0c7D45C3834E4eB18c26C60932B757A43B7B0B + Payee: 0xa3971A7234a3379A1813d9867B531e7EeB20ae07 + Service provider: 0xa3971A7234a3379A1813d9867B531e7EeB20ae07 + Provider: ezpdpz-calib (ID 3) + Commission: 0.00% + +Provider Service + Service URL: https://calib.ezpdpz.net + Min piece size: 1.0 KB + Max piece size: 32.0 GB + Storage price: < 0.0001 USDFC/TiB/month + Min proving period: 30 epochs + Location: unknown + Payment token: USDFC (native) + +Metadata + source: filecoin-pin + withIPFSIndexing: (empty) + +Pieces + Total pieces: 2 + Unique CommPs: 2 + Unique root CIDs: 2 + + #0 + CommP: bafkzcibcfab4grpgq6e6rva4kfuxfcvibdzx3kn2jdw6q3zqgwt5cou7j6k4wfq + Root CID: bafybeibh422kjvgfmymx6nr7jandwngrown6ywomk4vplayl4de2x553t4 + #1 + CommP: bafkzcibcjmcnyio2ocxhmtq34uh5ct425xzpnor532zku7tjvqf5toodbxtsqhi + Root CID: bafybeig27btater5fpt3l67gbme3sebqk3ynwdhlbrbuk3q7espiyplan4 +Data set inspection complete +``` + +**Key information:** + +**Data Set Status:** + +* **live** - Data set is active with ongoing PDP proofs +* **Pieces stored: 2** - Our demo.txt (#0) and my-data/ directory (#1) +* **Leaf count: 21** - Total Merkle tree leaves across all pieces +* **Total size: 672.0 B** - Combined size of both pieces + +**Payment Rails:** + +* **PDP rail ID: 631** - Active payment rail for storage proofs +* **Payer/Payee** - Payment flows from your wallet to the provider +* **Commission: 0.00%** - No commission on this testnet provider + +**Provider Details:** + +* **Service URL**: Direct access to download pieces +* **Storage price**: < 0.0001 USDFC/TiB/month +* **Min proving period: 30 epochs** - Proofs submitted every 15 minutes + +**Pieces:** + +* Each piece shows its CommP (Filecoin piece CID) and Root CID (IPFS content ID) +* Piece #0 = demo.txt (our single file upload) +* Piece #1 = my-data/ (our directory upload) + +> 💡 **Note**: This command queries the smart contracts on-chain to retrieve all data set information. The data shown is live blockchain state, not cached data. + +## Links + +### Documentation + +* [Frequently Asked Questions](faq.md) +* [Filecoin Pin Documentation](README.md) + +#### Repositories + +* [Filecoin Pin CLI](https://github.com/filecoin-project/filecoin-pin) +* [Filecoin Pin dApp](https://github.com/filecoin-project/filecoin-pin-website) + +#### Related Tools + +* [Synapse SDK](https://github.com/FilOzone/synapse-sdk) +* [USDFC Documentation](https://docs.secured.finance/usdfc-stablecoin/getting-started) diff --git a/build/cookbook/filecoin-pin/github-action.md b/build/cookbook/filecoin-pin/github-action.md new file mode 100644 index 000000000..62ed4f29b --- /dev/null +++ b/build/cookbook/filecoin-pin/github-action.md @@ -0,0 +1,17 @@ +--- +description: Host a static website with Filecoin Pin using GitHub Actions +--- + +# Filecoin Pin GitHub Action + +`filecoin-pin` can be used in CI pipelines like GitHub Actions to upload assets to the Filecoin decentralized storage network. Static website assets are particularly good candidates, given the existing tooling within the IPFS ecosystem for static website retrieval. + +The [Filecoin Pin repo](https://github.com/filecoin-project/filecoin-pin) has an [example upload to Filecoin GitHub Action](https://github.com/filecoin-project/filecoin-pin/tree/master/upload-action) that can be used directly or as a starting point for your own CI pipeline. + +The example filecoin-pin upload action itself has a [usage example](https://github.com/filecoin-project/filecoin-pin/tree/master/upload-action/examples), and you can even see it in production as part of [filecoin-pin-website's CI pipeline](https://github.com/filecoin-project/filecoin-pin-website/tree/main/.github/workflows)! + +Below is also a video walkthrough of the example GitHub Action in use! + +{% embed url="" %} + +Note: there is more work coming soon to add "filecoin-pin functionality" directly to the robust [ipshipyard/ipfs-deploy-action](https://github.com/ipshipyard/ipfs-deploy-action) ([tracking issue](https://github.com/ipshipyard/ipfs-deploy-action/issues/39)). diff --git a/getting-started/what-is-filecoin/crypto-economics.md b/getting-started/what-is-filecoin/crypto-economics.md index eccb4b1d2..4699c7daf 100644 --- a/getting-started/what-is-filecoin/crypto-economics.md +++ b/getting-started/what-is-filecoin/crypto-economics.md @@ -25,7 +25,7 @@ Additionally, 300 million FIL tokens are held in a mining reserve to incentivize ## Vesting -Mining rewards are subject to a vesting schedule to support long-term network alignment. For instance, 75% of block rewards earned by miners vest linearly over 180 days, while 25% are immediately accessible, improving miner cash flow and profitability. Note that if the miner has incurred "[fee debt](https://docs.filecoin.io/storage-providers/filecoin-economics/slashing)," the immediately accessible block rewards will automatically go towards paying down those fees. +Mining rewards are subject to a vesting schedule to support long-term network alignment. For instance, 75% of block rewards earned by miners vest linearly over 180 days, while 25% are immediately accessible, improving miner cash flow and profitability. Note that if the miner has incurred "[fee debt](../../storage-providers/filecoin-economics/slashing.md)," the immediately accessible block rewards will automatically go towards paying down those fees. A certain portion of initially printed FIL tokens are vested to Protocol Labs teams and the Filecoin Foundation over six years, and to SAFT investors over three years, as outlined in the [vesting schedule](https://observablehq.com/@starboard/a-primer-to-filecoin-circulating-supply/2). diff --git a/reference/built-in-actors/protocol-api.md b/reference/built-in-actors/protocol-api.md index 82e6cce62..fc27ea358 100644 --- a/reference/built-in-actors/protocol-api.md +++ b/reference/built-in-actors/protocol-api.md @@ -6,13 +6,13 @@ description: This page covers the Built-in actors Protocol API. The protocol level built-in actors API is split into the following sections: -* [Account actor](https://docs.filecoin.io/reference/built-in-actors/protocol-api/#account-actor) -* [Datacap](https://docs.filecoin.io/reference/built-in-actors/protocol-api/#datacap) -* [Miner](https://docs.filecoin.io/reference/built-in-actors/protocol-api/#miner) -* [Multisig](https://docs.filecoin.io/reference/built-in-actors/protocol-api/#multisig) -* [Storage market actor](https://docs.filecoin.io/reference/built-in-actors/protocol-api/#storage-market-actor) -* [Storage power actor](https://docs.filecoin.io/reference/built-in-actors/protocol-api/#storage-power-actor) -* [Verified registry actor](https://docs.filecoin.io/reference/built-in-actors/protocol-api/#verified-registry-actor) +* [Account actor](#account-actor) +* [Datacap](#datacap) +* [Miner](#miner) +* [Multisig](#multisig) +* [Storage market actor](#storage-market-actor) +* [Storage power actor](#storage-power-actor) +* [Verified registry actor](#verified-registry-actor) ## Account actor diff --git a/reference/general/glossary.md b/reference/general/glossary.md index f4f9a86d8..0c9c22a4d 100644 --- a/reference/general/glossary.md +++ b/reference/general/glossary.md @@ -16,19 +16,19 @@ In the Filecoin network, an _address_ is a unique cryptographic value that serve ## Block -In a blockchain, a _block_ is the fundamental unit of record. Each block is cryptographically linked to one or more previous blocks. Blocks typically contain [messages](https://docs.filecoin.io/reference/general/glossary/#message) relating changes to some state (for example, financial records) tracked by the blockchain. +In a blockchain, a _block_ is the fundamental unit of record. Each block is cryptographically linked to one or more previous blocks. Blocks typically contain [messages](#message) relating changes to some state (for example, financial records) tracked by the blockchain. ## Blockchain -Fundamentally, a _blockchain_ is a system of record in which new records, or [blocks](https://docs.filecoin.io/reference/general/glossary/#block) are cryptographically linked to preceding records. This construction is a foundational component of secure, verifiable, and distributed transaction ledgers. +Fundamentally, a _blockchain_ is a system of record in which new records, or [blocks](#block) are cryptographically linked to preceding records. This construction is a foundational component of secure, verifiable, and distributed transaction ledgers. ## Block height -The _height_ of a [block](https://docs.filecoin.io/reference/general/glossary/#block) corresponds to the number of [epochs](https://docs.filecoin.io/reference/general/glossary/#epoch) elapsed before the block was added to the blockchain. The height of the Filecoin [blockchain](https://docs.filecoin.io/reference/general/glossary/#blockchain) is defined to be the maximum height of any block in the blockchain. +The _height_ of a [block](#block) corresponds to the number of [epochs](#epoch) elapsed before the block was added to the blockchain. The height of the Filecoin [blockchain](#blockchain) is defined to be the maximum height of any block in the blockchain. ## Capacity commitment -If a storage provider doesn’t find any available deal proposals appealing, they can alternatively make a _capacity commitment_, filling a [sector](https://docs.filecoin.io/reference/general/glossary/#sector) with arbitrary data, rather than with client data. Maintaining this sector allows the storage provider to provably demonstrate that they are reserving space on behalf of the network. +If a storage provider doesn’t find any available deal proposals appealing, they can alternatively make a _capacity commitment_, filling a [sector](#sector) with arbitrary data, rather than with client data. Maintaining this sector allows the storage provider to provably demonstrate that they are reserving space on behalf of the network. ## CommP @@ -40,7 +40,7 @@ A self-describing format for referencing data in distributed information systems ## Collateral -In order to enter into a [storage deal](https://docs.filecoin.io/reference/general/glossary/#deal), a [storage provider](https://docs.filecoin.io/reference/general/glossary/#storage-provider) is required to provide [FIL](https://docs.filecoin.io/reference/general/glossary/#fil) as _collateral_, to be paid out as compensation to a client in the event that the provider fails to uphold their storage commitment. +In order to enter into a [storage deal](#deal), a [storage provider](#storage-provider) is required to provide [FIL](#fil) as _collateral_, to be paid out as compensation to a client in the event that the provider fails to uphold their storage commitment. ## Deal @@ -48,11 +48,11 @@ Two participants in the Filecoin network can enter into a _deal_ in which one pa ## Election -Every [epoch](https://docs.filecoin.io/reference/general/glossary/#epoch), a small subset of Filecoin [storage providers](https://docs.filecoin.io/reference/general/glossary/#storage-provider) are _elected_ to mine a new [block](https://docs.filecoin.io/reference/general/glossary/#block) for the Filecoin blockchain. A provider’s probability of being elected is roughly proportional to the share of the Filecoin network’s total storage capacity that they contribute. +Every [epoch](#epoch), a small subset of Filecoin [storage providers](#storage-provider) are _elected_ to mine a new [block](#block) for the Filecoin blockchain. A provider’s probability of being elected is roughly proportional to the share of the Filecoin network’s total storage capacity that they contribute. ## Epoch -Time in the Filecoin blockchain is discretized into _epochs_ that are currently thirty seconds in length. Every epoch, a subset of storage providers are elected to each add a new block to the Filecoin blockchain via [Winning Proof-of-Spacetime](https://docs.filecoin.io/reference/general/glossary/#winning-proof-of-spacetime-winningpost). +Time in the Filecoin blockchain is discretized into _epochs_ that are currently thirty seconds in length. Every epoch, a subset of storage providers are elected to each add a new block to the Filecoin blockchain via [Winning Proof-of-Spacetime](#winning-proof-of-spacetime-winningpost). ## FIL @@ -60,11 +60,11 @@ _FIL_ is the name of the Filecoin unit of currency; it is alternatively denoted ## Faucet -A _faucet_ is a service that provides free [FIL](https://docs.filecoin.io/reference/general/glossary/#fil). Typically, faucets are run for the benefit of new users in a network, providing them with the necessary seed capital to begin making transactions. +A _faucet_ is a service that provides free [FIL](#fil). Typically, faucets are run for the benefit of new users in a network, providing them with the necessary seed capital to begin making transactions. ## Fault -When a [storage provider](https://docs.filecoin.io/reference/general/glossary/#storage-provider) fails to complete [Window Proof-of-Spacetime](https://docs.filecoin.io/reference/general/glossary/#window-proof-of-spacetime-windowpost) for a given sector, the Filecoin network registers a _fault_ for that sector, and the provider is [_slashed_](https://docs.filecoin.io/reference/general/glossary/#slash). If a storage provider does not resolve the fault quickly, the network assumes they have abandoned their commitment. +When a [storage provider](#storage-provider) fails to complete [Window Proof-of-Spacetime](#window-proof-of-spacetime-windowpost) for a given sector, the Filecoin network registers a _fault_ for that sector, and the provider is [_slashed_](#slash). If a storage provider does not resolve the fault quickly, the network assumes they have abandoned their commitment. ## Filecoin @@ -72,11 +72,11 @@ The term _Filecoin_ is used generically to refer to the Filecoin project, protoc ## Finality -Finality refers to the immutability of messages and state recorded to the Filecoin blockchain. As new blocks are added to the blockchain, it becomes more and more difficult for older blocks to be altered, until they become effectively impossible to modify. The _finality period_ is the amount of time that must elapse before a block is considered completely immutable. In the current [mainnet](https://docs.filecoin.io/reference/general/glossary/#mainnet), this is configured as 900 [epochs](https://docs.filecoin.io/reference/general/glossary/#epoch). +Finality refers to the immutability of messages and state recorded to the Filecoin blockchain. As new blocks are added to the blockchain, it becomes more and more difficult for older blocks to be altered, until they become effectively impossible to modify. The _finality period_ is the amount of time that must elapse before a block is considered completely immutable. In the current [mainnet](#mainnet), this is configured as 900 [epochs](#epoch). ## Gas -_Gas_ is a property of a [message](https://docs.filecoin.io/reference/general/glossary/#message), corresponding to the resources involved in including that message in a given [block](https://docs.filecoin.io/reference/general/glossary/#block). For each message included in a block, the block’s creator extracts a fee from the message’s sender; this fee is proportional to the message’s gas. +_Gas_ is a property of a [message](#message), corresponding to the resources involved in including that message in a given [block](#block). For each message included in a block, the block’s creator extracts a fee from the message’s sender; this fee is proportional to the message’s gas. ## Mainnet @@ -86,7 +86,7 @@ If used as a proper noun, capitalize the term: _“I am providing on Mainnet.” ## Message -The term _message_ is used to refer to data stored as part of a [block](https://docs.filecoin.io/reference/general/glossary/#block). A block can contain several messages. +The term _message_ is used to refer to data stored as part of a [block](#block). A block can contain several messages. ## Merkle Directed Acyclic Graph @@ -100,11 +100,11 @@ Merkle DAGs are a fundamental component for the representation of relationships ## Miner -The Filecoin project uses the term _provider_ to refer to participants in the network who provide a service of value to a client. Other blockchains, like Ethereum and Bitcoin, use the term _miner_. At present, the Filecoin specification recognizes two provider types: [storage providers](https://docs.filecoin.io/reference/general/glossary/#storage-provider) and [retrieval providers](https://docs.filecoin.io/reference/general/glossary/#retrieval-provider). +The Filecoin project uses the term _provider_ to refer to participants in the network who provide a service of value to a client. Other blockchains, like Ethereum and Bitcoin, use the term _miner_. At present, the Filecoin specification recognizes two provider types: [storage providers](#storage-provider) and [retrieval providers](#retrieval-provider). ## Pledged storage -Storage capacity that a provider has promised to reserve for the Filecoin network via [Proof-of-Replication](https://docs.filecoin.io/reference/general/glossary/#proof-of-replication-porep) is termed _pledged storage_. +Storage capacity that a provider has promised to reserve for the Filecoin network via [Proof-of-Replication](#proof-of-replication-porep) is termed _pledged storage_. ## Proof-of-Storage @@ -116,23 +116,23 @@ _Note_: “proof” here is used in an informal sense - typically, these proofs ## Proof-of-Replication (PoRep) -_Proof-of-Replication_ is a procedure by which a [storage provider](https://docs.filecoin.io/reference/general/glossary/#storage-provider) can prove to the Filecoin network that they have created a unique copy of some piece of data on the network’s behalf. +_Proof-of-Replication_ is a procedure by which a [storage provider](#storage-provider) can prove to the Filecoin network that they have created a unique copy of some piece of data on the network’s behalf. ## Proof-of-Spacetime (PoSt) -_Proof-of-Spacetime_ is a procedure by which a [storage-provider](https://docs.filecoin.io/reference/general/glossary/#storage-provider) can prove to the Filecoin network they continue to store a unique copy of some data on behalf of the network. Proof-of-Spacetime manifests in two distinct varieties in the present Filecoin specification: [Window Proof-of-Spacetime](https://docs.filecoin.io/reference/general/glossary/#window-proof-of-spacetime-windowpost) and [Winning Proof-of-Spacetime](https://docs.filecoin.io/reference/general/glossary/#winning-proof-of-spacetime-winningpost). +_Proof-of-Spacetime_ is a procedure by which a [storage-provider](#storage-provider) can prove to the Filecoin network they continue to store a unique copy of some data on behalf of the network. Proof-of-Spacetime manifests in two distinct varieties in the present Filecoin specification: [Window Proof-of-Spacetime](#window-proof-of-spacetime-windowpost) and [Winning Proof-of-Spacetime](#winning-proof-of-spacetime-winningpost). ## Quality-adjusted storage power -The storage power a [storage provider](https://docs.filecoin.io/reference/general/glossary/#storage-provider) earns from a storage deal offered by a [verified client](https://docs.filecoin.io/reference/general/glossary/#verified-client) will be augmented by a multiplier. Power totals that take into account this multiplier are termed _quality adjusted_. +The storage power a [storage provider](#storage-provider) earns from a storage deal offered by a [verified client](#verified-client) will be augmented by a multiplier. Power totals that take into account this multiplier are termed _quality adjusted_. ## Retrieval provider -A _retrieval provider_ is a Filecoin participant that enters retrieval [deals](https://docs.filecoin.io/reference/general/glossary/#deal) with clients, agreeing to supply a client with a particular file in exchange for [FIL](https://docs.filecoin.io/reference/general/glossary/#fil). Note that unlike [storage providers](https://docs.filecoin.io/reference/general/glossary/#storage-provider), retrieval providers are not additionally rewarded with the ability to add blocks to the Filecoin blockchain; their only reward is the fee they extract from the client. +A _retrieval provider_ is a Filecoin participant that enters retrieval [deals](#deal) with clients, agreeing to supply a client with a particular file in exchange for [FIL](#fil). Note that unlike [storage providers](#storage-provider), retrieval providers are not additionally rewarded with the ability to add blocks to the Filecoin blockchain; their only reward is the fee they extract from the client. ## Seal -_Sealing_ is one of the fundamental building blocks of the Filecoin protocol. It is a computation-intensive process performed over a [sector](https://docs.filecoin.io/reference/general/glossary/#sector) that results in a unique representation of the sector. The properties of this new representation are essential to the [Proof-of-Replication](https://docs.filecoin.io/reference/general/glossary/#proof-of-replication-porep) and the [Proof-of-Spacetime](https://docs.filecoin.io/reference/general/glossary/#proof-of-spacetime-post) procedures. +_Sealing_ is one of the fundamental building blocks of the Filecoin protocol. It is a computation-intensive process performed over a [sector](#sector) that results in a unique representation of the sector. The properties of this new representation are essential to the [Proof-of-Replication](#proof-of-replication-porep) and the [Proof-of-Spacetime](#proof-of-spacetime-post) procedures. ## Sector @@ -140,15 +140,15 @@ Storage providers store data on behalf of the Filecoin network in fixed-size blo ## Slash -When a [fault](https://docs.filecoin.io/reference/general/glossary/#fault) is registered for a [sector](https://docs.filecoin.io/reference/general/glossary/#sector), the Filecoin network will _slash_ the [storage provider](https://docs.filecoin.io/reference/general/glossary/#storage-provider) that is supposed to be storing the sector; that is, it will assess penalties to the provider (to be paid out of the [collateral](https://docs.filecoin.io/reference/general/glossary/#collateral) fronted by the provider) for their failure to uphold their pledge of storage. When slashing takes place, the power a provider earns for the associated sector is subtracted from the provider’s total power for the purposes of [election](https://docs.filecoin.io/reference/general/glossary/#election). +When a [fault](#fault) is registered for a [sector](#sector), the Filecoin network will _slash_ the [storage provider](#storage-provider) that is supposed to be storing the sector; that is, it will assess penalties to the provider (to be paid out of the [collateral](#collateral) fronted by the provider) for their failure to uphold their pledge of storage. When slashing takes place, the power a provider earns for the associated sector is subtracted from the provider’s total power for the purposes of [election](#election). ## Storage provider -A _storage provider_ is a Filecoin participant that stores data on behalf of the network. Storage providers are rewarded for this service through payments by clients that contract their services, as well as by periodic authorization to extend the Filecoin [blockchain](https://docs.filecoin.io/reference/general/glossary/#blockchain) with [blocks](https://docs.filecoin.io/reference/general/glossary/#block) of their own creation. When they create a block, storage providers are rewarded with newly minted [FIL](https://docs.filecoin.io/reference/general/glossary/#fil), as well as the transaction fees they can levy on other participants seeking to include [messages](https://docs.filecoin.io/reference/general/glossary/#message) in the block. +A _storage provider_ is a Filecoin participant that stores data on behalf of the network. Storage providers are rewarded for this service through payments by clients that contract their services, as well as by periodic authorization to extend the Filecoin [blockchain](#blockchain) with [blocks](#block) of their own creation. When they create a block, storage providers are rewarded with newly minted [FIL](#fil), as well as the transaction fees they can levy on other participants seeking to include [messages](#message) in the block. ## Storage power -A [storage provider’s](https://docs.filecoin.io/reference/general/glossary/#storage-provider) _storage power_ is a value roughly proportional to the amount of storage capacity they make available on behalf of the network via [capacity commitments](https://docs.filecoin.io/reference/general/glossary/#capacity-commitment) or [storage deals](https://docs.filecoin.io/reference/general/glossary/#deal). Storage power is used to select storage providers for rewards in proportion to their contributions to the total network storage capacity. +A [storage provider’s](#storage-provider) _storage power_ is a value roughly proportional to the amount of storage capacity they make available on behalf of the network via [capacity commitments](#capacity-commitment) or [storage deals](#deal). Storage power is used to select storage providers for rewards in proportion to their contributions to the total network storage capacity. ## Zero-knowledge succinct non-interactive argument of knowledge (zk-SNARK) @@ -158,7 +158,7 @@ An _argument of knowledge_ is a construction by which one party, called the _pro * A _zero-knowledge_ argument of knowledge has the requirement that the verifier should not need access to the knowledge the prover has access to in order to verify the prover’s claim. * A _succinct_ argument of knowledge is one that can be “quickly” verified, and which is “small”, for appropriate definitions of both of those terms. -A zero-knowledge, succinct non-interactive argument of knowledge (zk-SNARK) embodies all of these properties. Filecoin utilizes these constructions to enable its distributed network to efficiently verify that [storage providers](https://docs.filecoin.io/reference/general/glossary/#storage-provider) are storing files they pledged to store, without requiring the verifiers to maintain copies of these files themselves. +A zero-knowledge, succinct non-interactive argument of knowledge (zk-SNARK) embodies all of these properties. Filecoin utilizes these constructions to enable its distributed network to efficiently verify that [storage providers](#storage-provider) are storing files they pledged to store, without requiring the verifiers to maintain copies of these files themselves. ## Testnet @@ -168,25 +168,25 @@ Note: if used as a proper noun, capitalize the term. For example, “I am provid ## Tipset -A [tipset](https://filecoin.io/blog/tipsets-family-based-approach-to-consensus/) is a set of [blocks](https://docs.filecoin.io/reference/general/glossary/#block) that each have the same [height](https://docs.filecoin.io/reference/general/glossary/#block-height) and parent tipset; the Filecoin [blockchain](https://docs.filecoin.io/reference/general/glossary/#blockchain) is a chain of tipsets, rather than a chain of blocks. +A [tipset](https://filecoin.io/blog/tipsets-family-based-approach-to-consensus/) is a set of [blocks](#block) that each have the same [height](#block-height) and parent tipset; the Filecoin [blockchain](#blockchain) is a chain of tipsets, rather than a chain of blocks. Each tipset is assigned a weight corresponding to the amount of storage the network is provided per the commitments encoded in the tipset’s blocks. The consensus protocol of the network directs nodes to build on top of the heaviest chain. -By basing its blockchain on tipsets, Filecoin can allow multiple [storage providers](https://docs.filecoin.io/reference/general/glossary/#storage-provider) to create blocks in the same [epoch](https://docs.filecoin.io/reference/general/glossary/#epoch), increasing network throughput. By construction, this also provides network security: a node that attempts to intentionally prevent the valid blocks of a second node from making it onto the canonical chain runs up against the consensus preference for heavier chains. +By basing its blockchain on tipsets, Filecoin can allow multiple [storage providers](#storage-provider) to create blocks in the same [epoch](#epoch), increasing network throughput. By construction, this also provides network security: a node that attempts to intentionally prevent the valid blocks of a second node from making it onto the canonical chain runs up against the consensus preference for heavier chains. ## Verified client -To further incentivize the storage of “useful” data over simple [capacity commitments](https://docs.filecoin.io/reference/general/glossary/#capacity-commitment), [storage providers](https://docs.filecoin.io/reference/general/glossary/#storage-provider) have the additional opportunity to compete for special [deals](https://docs.filecoin.io/reference/general/glossary/#deal) offered by [verified clients](https://docs.filecoin.io/reference/general/glossary/#verified-client). Such clients are certified with respect to their intent to offer deals involving the storage of meaningful data, and the power a storage provider earns for these deals is augmented by a multiplier. +To further incentivize the storage of “useful” data over simple [capacity commitments](#capacity-commitment), [storage providers](#storage-provider) have the additional opportunity to compete for special [deals](#deal) offered by [verified clients](#verified-client). Such clients are certified with respect to their intent to offer deals involving the storage of meaningful data, and the power a storage provider earns for these deals is augmented by a multiplier. ## Window Proof-of-Spacetime (WindowPoSt) -_Window Proof-of-Spacetime_ (WindowPoSt) is the mechanism by which the commitments made by [storage providers](https://docs.filecoin.io/reference/general/glossary/#storage-provider) are audited. It sees each 24 hour period broken down into a series of windows. Correspondingly, each storage provider’s set of pledged [sectors](https://docs.filecoin.io/reference/general/glossary/#sector) is partitioned into subsets, one subset for each window. Within a given window, each storage provider must submit a [Proof-of-Spacetime](https://docs.filecoin.io/reference/general/glossary/#proof-of-spacetime-post) for each sector in their respective subset. This requires ready access to each of the challenged sectors, and will result in a proof compressed via [zk-SNARK](https://docs.filecoin.io/reference/general/glossary/#zero-knowledge-succinct-non-interactive-argument-of-knowledge-zk-snark) being published to the Filecoin [blockchain](https://docs.filecoin.io/reference/general/glossary/#blockchain) as a [message](https://docs.filecoin.io/reference/general/glossary/#message) in a [block](https://docs.filecoin.io/reference/general/glossary/#block). In this way, every sector of [pledged storage](https://docs.filecoin.io/reference/general/glossary/#pledged-storage) is audited at least once in any 24 hour period, and a permanent, verifiable, and public record attesting to each storage provider’s continued commitment is kept. +_Window Proof-of-Spacetime_ (WindowPoSt) is the mechanism by which the commitments made by [storage providers](#storage-provider) are audited. It sees each 24 hour period broken down into a series of windows. Correspondingly, each storage provider’s set of pledged [sectors](#sector) is partitioned into subsets, one subset for each window. Within a given window, each storage provider must submit a [Proof-of-Spacetime](#proof-of-spacetime-post) for each sector in their respective subset. This requires ready access to each of the challenged sectors, and will result in a proof compressed via [zk-SNARK](#zero-knowledge-succinct-non-interactive-argument-of-knowledge-zk-snark) being published to the Filecoin [blockchain](#blockchain) as a [message](#message) in a [block](#block). In this way, every sector of [pledged storage](#pledged-storage) is audited at least once in any 24 hour period, and a permanent, verifiable, and public record attesting to each storage provider’s continued commitment is kept. -The Filecoin network expects constant availability of stored data. Failing to submit WindowPoSt for a sector will result in a [fault](https://docs.filecoin.io/reference/general/glossary/#fault), and the storage provider supplying the sector will be [slashed](https://docs.filecoin.io/reference/general/glossary/#slash). +The Filecoin network expects constant availability of stored data. Failing to submit WindowPoSt for a sector will result in a [fault](#fault), and the storage provider supplying the sector will be [slashed](#slash). ## Winning Proof-of-Spacetime (WinningPoSt) -_Winning Proof-of-Spacetime_ (WinningPoSt) is the mechanism by which [storage providers](https://docs.filecoin.io/reference/general/glossary/#storage-provider) are rewarded for their contributions to the Filecoin network. At the beginning of each [epoch](https://docs.filecoin.io/reference/general/glossary/#epoch), a small number of storage providers are [elected](https://docs.filecoin.io/reference/general/glossary/#election) to each mine a new [block](https://docs.filecoin.io/reference/general/glossary/#block). As a requirement for doing so, each provider is tasked with submitting a compressed [Proof-of-Storage](https://docs.filecoin.io/reference/general/glossary/#proof-of-storage) for a specified [sector](https://docs.filecoin.io/reference/general/glossary/#sector). Each elected provider who successfully creates a block is granted [FIL](https://docs.filecoin.io/reference/general/glossary/#fil), as well as the opportunity to charge other Filecoin participants fees to include [messages](https://docs.filecoin.io/reference/general/glossary/#message) in the block. +_Winning Proof-of-Spacetime_ (WinningPoSt) is the mechanism by which [storage providers](#storage-provider) are rewarded for their contributions to the Filecoin network. At the beginning of each [epoch](#epoch), a small number of storage providers are [elected](#election) to each mine a new [block](#block). As a requirement for doing so, each provider is tasked with submitting a compressed [Proof-of-Storage](#proof-of-storage) for a specified [sector](#sector). Each elected provider who successfully creates a block is granted [FIL](#fil), as well as the opportunity to charge other Filecoin participants fees to include [messages](#message) in the block. Storage providers who fail to do this in the necessary window will forfeit their opportunity to mine a block, but will not otherwise incur penalties for their failure to do so. diff --git a/storage-providers/architecture/lotus-components.md b/storage-providers/architecture/lotus-components.md index 28062e241..da68c8e34 100644 --- a/storage-providers/architecture/lotus-components.md +++ b/storage-providers/architecture/lotus-components.md @@ -31,7 +31,7 @@ The machine running the Lotus daemon must be connected to the public internet fo #### Syncing the chain -Syncing the chain is a key role of the daemon. It communicates with the other nodes on the network by sending messages, which are, in turn, collected into [blocks](https://docs.filecoin.io/reference/general/glossary/#block). These blocks are then collected into [tipsets](https://docs.filecoin.io/reference/general/glossary/#tipset). Your Lotus daemon receives the messages on-chain, enabling you to maintain consensus about the state of the Filecoin network with all the other participants. +Syncing the chain is a key role of the daemon. It communicates with the other nodes on the network by sending messages, which are, in turn, collected into [blocks](../../reference/general/glossary.md#block). These blocks are then collected into [tipsets](../../reference/general/glossary.md#tipset). Your Lotus daemon receives the messages on-chain, enabling you to maintain consensus about the state of the Filecoin network with all the other participants. Due to the growth in the size of the chain since its genesis, it is not advised for storage providers to sync the entire history of the network. Instead, providers should use the available [lightweight snapshots](https://forest-archive.chainsafe.dev/list) to import the most recent messages. One exception in which a provider would need to sync the entire chain would be to run a blockchain explorer. @@ -92,7 +92,7 @@ Another key responsibility of the Lotus Miner is the scheduling of jobs for the #### Storage proving -One of the most important roles of `lotus-miner` is the Storage proving. Both [WindowPoSt](https://docs.filecoin.io/reference/general/glossary/#window-proof-of-spacetime-windowpost) and [WinningPoSt](https://docs.filecoin.io/reference/general/glossary/#winning-proof-of-spacetime-winningpost) processes are usually handled by the `lotus-miner` process. For scalability and reliability purposes it is now also possible to run these proving processes on dedicated servers (proving workers) instead of using the Lotus miner. +One of the most important roles of `lotus-miner` is the Storage proving. Both [WindowPoSt](../../reference/general/glossary.md#window-proof-of-spacetime-windowpost) and [WinningPoSt](../../reference/general/glossary.md#winning-proof-of-spacetime-winningpost) processes are usually handled by the `lotus-miner` process. For scalability and reliability purposes it is now also possible to run these proving processes on dedicated servers (proving workers) instead of using the Lotus miner. The proving processes require low-latency access to sealed sectors. The proving challenge requires a GPU to run on. The resulting `zkProof` will be sent to the chain in a message. Messages must arrive within 30 minutes for WindowPoSt, and 30 seconds for WinningPoSt. It is extremely important that providers properly size and configure the proving workers, whether they are using just the Lotus miner or separate workers. Additionally, dedicated wallets, described in Control wallets, should be set up for these processes. diff --git a/storage-providers/infrastructure/backup-and-disaster-recovery.md b/storage-providers/infrastructure/backup-and-disaster-recovery.md index 935e4815d..ad2c610e5 100644 --- a/storage-providers/infrastructure/backup-and-disaster-recovery.md +++ b/storage-providers/infrastructure/backup-and-disaster-recovery.md @@ -66,6 +66,7 @@ This helps to reduce the amount of manual tasks for a failover drastically. If t Having the services on a floating IP allows to assign this IP to another machine and start the service on it. ## No Penalty for Recovered Faults -Note that as of [FIP006: No repay debt requirement for DeclareFaultsRecovered](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0006.md), storage providers are no longer required to pay [fee debt](https://docs.filecoin.io/storage-providers/filecoin-economics/slashing) prior to recovering a new storage fault. This enables a storage provider that currently has accrued fee debt to recover faults without being further penalized with additional fees. + +Note that as of [FIP006: No repay debt requirement for DeclareFaultsRecovered](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0006.md), storage providers are no longer required to pay [fee debt](../filecoin-economics/slashing.md) prior to recovering a new storage fault. This enables a storage provider that currently has accrued fee debt to recover faults without being further penalized with additional fees. [Was this page helpful?](https://airtable.com/apppq4inOe4gmSSlk/pagoZHC2i1iqgphgl/form?prefill\_Page+URL=https://docs.filecoin.io/storage-providers/infrastructure/backup-and-disaster-recovery)