LLMs.txt Explorer
Discovers and fetches llms.txt files from websites, which contain instructions for how AI models should interact with those sites.
Provides a bridge for exploring websites with llms.txt files, enabling fetching and parsing of website-specific language model instructions for context-aware interactions in web-based environments.
What it does
- Check websites for llms.txt files
- Parse and validate llms.txt content
- List known websites with llms.txt files
- Filter results by file type
Best for
About LLMs.txt Explorer
LLMs.txt Explorer is a community-built MCP server published by thedaviddias that provides AI assistants with tools and capabilities via the Model Context Protocol. Explore websites using llms.txt files with LLMs.txt Explorer. Fetch and parse site-specific language model instructions for smarter web interactions. It is categorized under search web.
How to install
You can install LLMs.txt Explorer in your AI client of choice. Use the install panel on this page to get one-click setup for Cursor, Claude Desktop, VS Code, and other MCP-compatible clients. This server runs locally on your machine via the stdio transport.
License
LLMs.txt Explorer is released under the NOASSERTION license.
MCP LLMS.txt Explorer
A Model Context Protocol server for exploring websites with llms.txt files. This server helps you discover and analyze websites that implement the llms.txt standard.
Features
Resources
- Check websites for llms.txt and llms-full.txt files
- Parse and validate llms.txt file contents
- Access structured data about compliant websites
Tools
check_website- Check if a website has llms.txt files- Takes domain URL as input
- Returns file locations and validation status
list_websites- List known websites with llms.txt files- Returns structured data about compliant websites
- Supports filtering by file type (llms.txt/llms-full.txt)
Development
Install dependencies:
pnpm install
Build the server:
pnpm run build
For development with auto-rebuild:
pnpm run watch
Installation
Installing via Smithery
To install mcp-llms-txt-explorer for Claude Desktop automatically via Smithery:
npx -y @smithery/cli install @thedaviddias/mcp-llms-txt-explorer --client claude
Installing Manually
To use this server:
# Clone the repository
git clone https://github.com/thedaviddias/mcp-llms-txt-explorer.git
cd mcp-llms-txt-explorer
# Install dependencies
pnpm install
# Build the server
pnpm run build
Configuration with Claude Desktop
To use with Claude Desktop, add the server config:
On MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json
On Windows: %APPDATA%/Claude/claude_desktop_config.json
{
"mcpServers": {
"llms-txt-explorer": {
"command": "node",
"args": ["/path/to/llms-txt-explorer/build/index.js"]
}
}
}
For npx usage, you can use:
{
"mcpServers": {
"llms-txt-explorer": {
"command": "npx",
"args": ["-y", "@thedaviddias/mcp-llms-txt-explorer"]
}
}
}
Debugging
Since MCP servers communicate over stdio, debugging can be challenging. We recommend using the MCP Inspector, which is available as a package script:
pnpm run inspector
The Inspector will provide a URL to access debugging tools in your browser.
License
This project is licensed under the MIT License—see the LICENSE file for details.
Alternatives
Related Skills
Browse all skillsOfficial Google SEO guide covering search optimization, best practices, Search Console, crawling, indexing, and improving website search visibility based on official Google documentation
Optimize Core Web Vitals (LCP, INP, CLS) for better page experience and search ranking. Use when asked to "improve Core Web Vitals", "fix LCP", "reduce CLS", "optimize INP", "page experience optimization", or "fix layout shifts".
Create user-centered, accessible interface copy (microcopy) for digital products including buttons, labels, error messages, notifications, forms, onboarding, empty states, success messages, and help text. Use when writing or editing any text that appears in apps, websites, or software interfaces, designing conversational flows, establishing voice and tone guidelines, auditing product content for consistency and usability, reviewing UI strings, or improving existing interface copy. Applies UX writing best practices based on four quality standards — purposeful, concise, conversational, and clear. Includes accessibility guidelines, research-backed benchmarks (sentence length, comprehension rates, reading levels), expanded error patterns, tone adaptation frameworks, and comprehensive reference materials.
This skill should be used when users need to search the web for information, find current content, look up news articles, search for images, or find videos. It uses DuckDuckGo's search API to return results in clean, formatted output (text, markdown, or JSON). Use for research, fact-checking, finding recent information, or gathering web resources.
Automate web browser interactions using natural language via CLI commands. Use when the user asks to browse websites, navigate web pages, extract data from websites, take screenshots, fill forms, click buttons, or interact with web applications. Triggers include "browse", "navigate to", "go to website", "extract data from webpage", "screenshot", "web scraping", "fill out form", "click on", "search for on the web". When taking actions be as specific as possible.
Manage Zotero reference libraries via the Web API. Search, list, add items by DOI/ISBN/PMID (with duplicate detection), delete/trash items, update metadata and tags, export in BibTeX/RIS/CSL-JSON, batch-add from files, check PDF attachments, cross-reference citations, find missing DOIs via CrossRef, and fetch open-access PDFs. Supports --json output for scripting. Use when the user asks about academic references, citation management, literature libraries, PDFs for papers, bibliography export, or Zotero specifically.