I Tested Grabby with Passmark. One Case Failed

I came across Grabby while looking for a lightweight way to extract readable markdown from web pages. It is an open-source browser tool: you give it a URL, it fetches the page, and it spits out markdown you can download. It looks simple, but it still takes user input, and that is usually where things get interesting.

I decided to write a structured test suite for Grabby using Passmark, a browser-based testing tool, partly to see how the app held up under different conditions, and partly to practice writing a suite that goes beyond "does the happy path work?"

What Grabby does

You open the app, paste in a URL, and click a button labelled "Extract Markdown." Grabby fetches the page, strips the HTML, and returns markdown. There is also a download button so you can save the result.
The source is on GitHub, and the app is deployed on Netlify.

The app takes user-provided URLs, does something with them, and renders output. That means you need to think about what happens with bad URLs and unsafe input, especially the cases the happy path would never reveal.

What I tested

I focused on how the app behaves when the input is not ideal. That includes:

• invalid URLs

• unsafe input

• complex pages

• whether the output is actually usable

Why I used Passmark

Passmark lets you write browser automation tests with a straightforward API. You describe what the browser should do and what it should assert. I picked it for this because the tests are readable, the setup is light, and i wanted something i could share alongside the article without asking readers to spin up a local environment.

I ran everything against the live Netlify deployment so the results reflect what a real user would see, not a local setup that behaves differently.

How I designed the suite

I did not want to write only one happy-path test. Grabby accepts user-provided URLs, so the risky parts are around input handling, page rendering, and what happens when the app receives something unexpected.

I grouped the suite into:

• normal extraction

• invalid input

• unsafe input

• complex page handling

• download behavior

Each group focuses on a different failure point. Normal extraction checks that the core feature works. Invalid input checks what happens when you give the app something that is not a URL. Unsafe input checks whether the app does anything dangerous with adversarial values. Complex page handling checks whether Grabby degrades gracefully on JavaScript-heavy pages. Download behavior checks that the output is actually retrievable.

The tests, group by group

Normal extraction

For this baseline test, i pointed the app at https://httpbin.org/html, a stable page with predictable HTML output, clicked the button, and asserted that a markdown file appeared with a .md filename and a visible Download button.

test("Happy path: extracts markdown from a valid URL", async ({ page }) => {
  test.setTimeout(90_000);

  await runSteps({
    page,
    userFlow: "Extract markdown from a valid URL",
    steps: [
      { description: `Navigate to ${APP_URL}` },
      {
        description: "Enter a valid website URL into the Website URL input",
        data: { value: "https://httpbin.org/html" },
      },
      {
        description: "Click the Extract Markdown button",
        waitUntil: "A markdown file result is displayed",
      },
    ],
    assertions: [
      {
        assertion:
          "A markdown file result is displayed with a filename ending in .md and a Download button is visible",
      },
    ],
    test,
    expect,
  });
});

This passed. Grabby fetched the page and returned markdown with the filename and download button both present. The output looked correct for that page.

Invalid input

I treated invalid input as its own group because this is where most apps start to break. I tested:

• plain text instead of a URL

• empty input

The first entered the string "this is not a website" into the URL field and clicked Extract. The assertion was that the app should not generate a markdown file and should show a clear validation message. The second left the field empty and expected the same outcome.

Both passed. The app rejected both inputs and did not attempt a fetch.

Unsafe input

This group had three tests. I tried:

• malformed URLs

• javascript: URLs

• raw script input

  I tested a malformed URL ("https://.") alongside javascript: and raw script input.

test("Security: handles script-like input safely", async ({ page }) => {
  test.setTimeout(60_000);

  await runSteps({
    page,
    userFlow: "Handle script injection attempt safely",
    steps: [
      { description: `Navigate to ${APP_URL}` },
      {
        description: "Enter a script tag into the Website URL input",
        data: { value: "<script>alert('xss')</script>" },
      },
      { description: "Click the Extract Markdown button" },
    ],
    assertions: [
      {
        assertion:
          "The app does not execute the script, does not render it as HTML, and shows a validation error",
      },
    ],
    test,
    expect,
  });
});

All three passed. The app did not execute the script, did not render it as HTML, and rejected the malformed and javascript: inputs without generating any output.

Complex page handling

One test in this group: I pointed Grabby at https://threejs.org/examples/, a WebGL-heavy page with minimal readable text content. The assertion was that the app should handle it gracefully by either generating markdown or showing a clear message, without crashing.
This passed. Grabby returned something rather than hanging or throwing an error. The output was sparse given the nature of the page, but the app did not break.

Download behavior

After a successful extraction from https://example.com, i clicked the Download button and asserted that the file remained available and the page did not break.
This passed. Clicking the download button triggered a file download with the expected content.

The test that failed

test("Security: rejects local network URL", async ({ page }) => {
  test.setTimeout(120_000);

  await runSteps({
    page,
    userFlow: "Reject localhost URL extraction",
    steps: [
      { description: `Navigate to ${APP_URL}` },
      {
        description: "Enter a localhost URL into the Website URL input",
        data: { value: "http://localhost:3000" },
      },
    ],
    assertions: [
      {
        assertion:
          "The Extract Markdown button is disabled, preventing localhost or private internal URLs from being submitted",
      },
    ],
    test,
    expect,
  });
});

Notice the test does not even click the button. The assertion is that the button should be disabled the moment http://localhost:3000 appears in the input field. That is the behaviour i expected: detect a private address and block the action before anything gets submitted.

The button was not disabled and it stayed active. Clicking it would send a request to http://localhost:3000, whatever happens to be running there on the user's machine.

What the failed test actually means

This is a validation gap. The app handles obvious cases like empty input, but it does not treat private addresses differently.

It also creates a UX issue. If someone enters http://localhost:3000, there is no clear signal that the input is invalid.

There is also a security angle. If the fetch runs server-side, allowing private addresses opens the door to SSRF-style probing.

This is not a hard problem to fix. Validate the URL before the fetch and reject anything that resolves to a loopback address or private range. This needs to happen before the request is sent.

If i were fixing this, i would:

• block localhost and common private ranges before the request

• treat those inputs as invalid, not just failed requests

• show a clear message instead of letting it silently fail

What I would improve in the suite

The test i wrote checks whether the button is disabled, which assumes the validation happens at the UI level. A stronger version would check the network request - confirm that no request to localhost is ever sent, regardless of what the button does. That way, even if the UI blocks the action, the test would catch a case where the underlying request logic still fires.

I would also add tests for IPv4 private ranges and IPv6 loopback. http://127.0.0.1 behaves the same as http://localhost and is worth covering explicitly.

One more gap: the current suite does not check what the markdown output actually contains, only that it is not empty. For a more thorough suite, you would want to assert something about the structure of the output, for example that a known heading from the test page appears in the result.

Final thoughts

Grabby does what it says. The main flow works, and most edge cases hold up.

The localhost test was the most useful one. It exposed something i would not have noticed if i stopped at the happy path. I would not have caught it if i only tested the normal flow. That is where it falls short.

If you're curious about the full test suite, it's on GitHub. And if you want to try Grabby yourself, the app is at grabby-c.netlify.app.