Denwix SEO Crawler:
SEO Spider Software for Windows

What Our SEO Spider Software Does

Our SEO spider software crawls your site much like a search engine bot. You enter your home page URL, and it goes through every link it finds. As it does this, it records the status code, the page title, the headings and many other details for each URL. As a result, you see your entire site the way Google sees it, in a single table.

Most SEO tools check a single page of your site. A crawler, in contrast, checks every page and shows how they link to one another. With a crawler, you can find the broken link on page 400, a redirect nobody noticed, or a page with no inbound links.

Checks the SEO Crawler Runs on Every Page

The free version and the paid plans use the same audit. Therefore, the only limit on Free is the number of URLs in each crawl. The SEO crawler checks the following:

• Response codes: 2xx, 3xx, 4xx and 5xx answers, timeouts and blocked URLs.
• Redirects: every hop of a chain, loops, and 301 versus 302.
• Page titles and meta descriptions: the crawler flags any that are missing, repeated across URLs, cut off for length or too thin.
• Headings: missing or repeated H1 tags and skipped heading levels.
• Canonicals and indexing: canonical tags, meta robots, X-Robots-Tag and how they conflict with each other.
• Images: missing, empty or over-long alt text.
• Content: word count, thin pages and near-duplicate pages.
• Links: crawl depth, inlinks, orphan pages and internal PageRank.
• Structured data: JSON-LD that does not parse or misses required fields.

From a List of Problems to a Fix Plan

A list of 2,000 problems is hard to manage. For this reason, the crawler also creates a fix plan. It sorts the problems into groups and ranks each one by its impact on the site and the effort the fix takes. Then it explains every problem in clear words: what it is, why it matters and how to solve it.

You can export the plan as an HTML report, a JSON file or a PDF audit report. However, to make the PDF, you need Chrome or Edge on your computer. If you have neither, the app creates a page that you can print.

The AI Search Audit in Our SEO Spider Software

AI search tools like ChatGPT search and Perplexity use their own bots to read the web. If your robots.txt blocks those bots, AI answers cannot quote or cite your pages. Therefore, the crawler reads your robots.txt file the way the robots.txt standard (RFC 9309) describes it. Then it reports the status of all 34 known AI bots.

Next, it checks your llms.txt, the Content-Signal lines that Cloudflare adds to robots.txt, and your structured data. In addition, it scores each page by how easy its main text is to quote. All of this is simple parsing. The crawler calls no AI service, and none of your files leave your computer.

Model-Training Crawlers and AI Answer Crawlers

Each AI crawler arrives with its own purpose. For example, some collect text to train models, while others fetch pages to answer a user's question right away. Blocking a training bot can be a fair choice. However, blocking a search bot removes your pages from AI answers. The audit lists training bots and search bots separately. So you know whether each block was on purpose and whether it may cost you visitors.

When Page Text Appears Only After Scripts Run

Many AI bots cannot run JavaScript. So if most of your content loads through a script, those bots find an empty page. The crawler compares the raw HTML of each page with what a user sees in a browser. Then it flags the pages that may have a problem. Google does run JavaScript. Even so, its JavaScript SEO basics guide explains why server-side HTML is the safest choice.

How the Offline Licence Key Works

Denwix signs every licence key with an Ed25519 private key that only Denwix holds. The matching public key sits inside the app. This lets the app check your key on your computer each time it starts. It needs no login, no account and no call to a remote server.

Buying also works from within Denwix SEO Crawler itself. The app opens this page with a one-time claim code. Once you check out and the payment goes through, the app collects its key by itself. We also e-mail you the key, so you can paste it on another computer later.

Outbound Requests From Denwix SEO Crawler

Apart from the sites you crawl, the app contacts denwix.com for just two reasons. First, it checks for updates once a day when it opens. Second, after a purchase, it asks whether the key for your claim code is ready. The app does not track you, and it collects no telemetry.

Some Premium features use outside services, like PageSpeed Insights or an AI model. Every request goes straight from your computer to that service, and none goes through Denwix. Nothing AI-related runs until you paste in an API key of your own.

A Small Website Crawler Tool With Zero Add-Ons

We built the engine on Node.js without a single outside npm package. It relies only on what Node.js itself offers, such as the built-in SQLite database and fetch. Therefore, you do not need Java, Docker or third-party libraries that an attacker could replace with harmful code.

The app keeps your crawl data in a database file within your Windows user profile. So you can delete, rename, export or import your crawls whenever you need. And because this website crawler tool runs on your own computer, it can also crawl staging sites and pages behind a firewall.

System Requirements and Installation

The app needs a 64-bit version of Windows 10 (version 1809 or higher) or Windows 11. JavaScript rendering, screenshots and the PDF report also need Google Chrome or Microsoft Edge, and most PCs already have one. If you crawl hundreds of thousands of URLs, close other memory-heavy apps first.

The first time you start the installer, Windows SmartScreen may show the message "Windows protected your PC." That happens because the installer has no code signature yet. To go ahead, open More info on that screen and press the Run anyway button that shows up. The app installs into your own user profile, so you do not need admin rights.

Check the Installer With SHA-256

Since the file has no code signature, you can check it yourself. The download section near the top of this page shows the SHA-256 hash of the latest installer. In Command Prompt, run the certutil line given there and see whether the two hashes match. Should even one character differ, treat the installer as tampered with and delete it unopened.

How It Compares With Other SEO Spider Software

Screaming Frog SEO Spider is the most popular tool of this type, but its free version stops at 500 URLs. Our free version crawls up to 1,000 URLs. Also, every plan includes the AI search audit, the llms.txt check and a fix plan.

In fairness, though, Screaming Frog has been around for a long time. It has a large user base, many guides and a Mac version. Denwix SEO Crawler, on the other hand, is new and runs on Windows only. So the best way to choose is to crawl your own site with the free version of both tools and compare the results.

What the Crawler Does Not Do Yet

Before you buy, here is what is missing today. So far the crawler runs on Windows alone, with no macOS or Linux build. Its menus and dialogs are available in English only. Also, the Search Console, GA4, Moz, Majestic and Ahrefs connections are in beta, because we tested them with sample data rather than real accounts.

Licence Terms in Short

These terms apply to every licence, so you know exactly what you get before you pay.

• Length: each licence lasts one year from the day you buy it.
• Seats: Premium is for one person; Agency covers your whole team.
• White-label: Agency reports carry your logo, name and colour instead of ours.
• Updates: every new version is free to download.
• After one year: the app goes back to the free 1,000-URL limit, and all your saved crawls stay intact.
• Payments and refunds: every order follows our refund policy and our terms and conditions.

Rather Have Denwix Run the Audit for You?

If you would rather not crawl the site yourself, our technical SEO audit team runs the same checks and corrects the core files for you. And if you only want to look at one page, the free SEO checker is usually enough.