For agents: Read llms.txt for the complete documentation and public content index. Request this page as Markdown.

Skip to main content
Ceisium logo Ceisium

Crawl Logs

Track crawl status, page outcomes, failures, retries, and coverage.

Browse docs

Crawl Logs show what Ceisium Search found, indexed, skipped, or failed during a crawl.

Crawl Logs list recent crawl runs with status, timing, source scope, and page outcome counts.
Ceisium Crawl logs page showing source selection, run filters, crawl statuses, page counts, and recent Next.js App Router crawl jobs.

Crawl Logs list recent crawl runs with status, timing, source scope, and page outcome counts.

Run states

StateMeaning
QueuedCrawl is waiting to start.
RunningCeisium Search is fetching and indexing pages.
CompletedCrawl finished and page counts are ready.
FailedCrawl stopped before completing. Check the error and source settings.

Page counts

CountMeaning
DiscoveredURLs found during crawl.
IndexedPages added to search.
SkippedPages ignored because of scope, duplicate URLs, or unsupported content.
FailedPages Ceisium Search tried but could not fetch or index.
Open URL details to inspect individual indexed, skipped, and failed URLs for a crawl.
Ceisium URL details drawer showing indexed, skipped, failed, and total URL counts with a folder tree of indexed Next.js App Router documentation pages.

Open URL details to inspect individual indexed, skipped, and failed URLs for a crawl.

Next.js example

For a source that includes /docs/app and its child paths on https://nextjs.org, check for pages such as:

TXT
/docs/app/docs/app/getting-started/installation/docs/app/guides/json-ld/docs/app/api-reference/functions/use-params

If /docs/app/guides/json-ld is missing, check include and exclude paths before rerunning the crawl.

Debug flow

  1. Open the latest crawl.
  2. Filter URL events to Failed and Skipped.
  3. Check whether the missing page is outside the source path.
  4. Adjust source scope.
  5. Run crawl again.