Why can't my Codex CLI Agent browse the web?
Codex CLI agents can search the web by default, but the default mode is cached: results come from an OpenAI-maintained pre-indexed snapshot rather than fetching live pages. Web search used to be disabled by default in earlier versions, which is why many developers still add config flags that are no longer needed. Today, to get live results, set web_search = "live" in ~/.codex/config.toml or pass --search for a single run. Even in live mode, the built-in search returns snippets only and does not scrape page content, which limits what a web search tool for AI agents can actually do with the results.
| Capability | Codex cached (default) | Codex live (web_search = "live") | Firecrawl |
|---|---|---|---|
| Web search | Yes, pre-indexed snippets | Yes, live snippets | Yes, full results |
| Read page content | No | No | Yes, via scrape |
| Multi-page crawl | No | No | Yes |
| Structured extraction | No | No | Yes |
| Agent-optimized index | No | No | Yes |
Use Codex cached search for lightweight lookups where a pre-indexed snippet is enough and you want no external dependencies. Switch to live mode when your agent needs current data. Switch to Firecrawl when your agent needs to read the full content of a page, follow links across a site, or extract structured data reliably.
Firecrawl is built around an agent-first web index that serves clean, LLM-ready content rather than raw HTML, so agents get higher-quality results without post-processing. Pair it with Codex via the Firecrawl MCP or CLI to give your agent full web access in a few lines of config.
The step-by-step setup guide at Codex web search with Firecrawl covers both paths, including how to disable the stale cached default. For a curated list of the most useful Codex agent skills worth adding to your Codex CLI workflow, see the best Codex skills guide. If you're deciding whether to use Codex or switch to Claude Code, see the Claude Code vs Codex comparison for a side-by-side on web access, sandboxing, and pricing.