WithOne.AI knowledge base

Scrapper

Paginated crawl, action detail extraction, and Word export in one local Python script for every platform catalog.

View on GitHub Setup guide
250+ Word exports already in Tools/
Auto pages Pagination detected per catalog
Local CLI Chrome + Selenium on your machine
Clean docs Headings, tables, code preserved

How it works

From knowledge page to formatted document

Scrapper opens a WithOne.AI catalog, walks every list page, opens each action, then writes a structured Word file.

  1. 01

    Open the catalog

    Load withone.ai/knowledge/{platform} in Chrome and wait for the action list.

  2. 02

    Walk pagination

    Detect total pages from the UI, advance through results, and stop when the list ends.

  3. 03

    Extract action detail

    Capture titles, HTTP methods, endpoints, parameters, tables, and code samples.

  4. 04

    Write the Word export

    Save to Tools/{platform}-tools.docx with headings, lists, monospace code, and tables.

Capabilities

Built for noisy knowledge pages

Heading heuristics, noise filters, and table detection keep exports readable without hand cleanup.

Smart pagination

Infers how many list pages exist for a platform and walks them until results stop. No hardcoded page caps.

Method aware extraction

Keeps GET, POST, PUT, PATCH, and DELETE visible next to endpoints so catalogs stay scannable in Word.

Content hygiene

Strips nav chrome, suggest-edit prompts, and copy-code UI while preserving real code blocks and language tags.

Heading heuristics

Promotes short Title Case lines only when followed by substantive content. Code-like prefixes never become headings.

Table rendering

Detects pipe, tab, and multi-space columns, then writes them as Word tables with bold header rows.

Stable navigation

Uses direct URLs between list and detail pages instead of browser back, avoiding stale document state.

Catalog coverage

Hundreds of platform exports ready

Point PlatformScraper at any WithOne.AI knowledge slug. The Tools folder already holds prior runs.

Development

GitHub, GitLab, Linear, Jira, Trello, Asana, Monday, Confluence

Communication

Slack, Discord, Intercom, Zoom, Google Meet

Productivity

Notion, Google Drive, Google Docs, Figma, Loom

Business

HubSpot, Salesforce, Zendesk, Datadog, PagerDuty, Stripe

Local setup

Running in minutes

Requires Python 3.10+, Google Chrome, and the packages in requirements.txt. No API keys needed for public pages.

1

Clone the repo and create a virtualenv

2

Install dependencies from requirements.txt

3

Run python scraper.py (optional --max-pages / --max-actions)

git clone https://github.com/Bixal99/Scrapper.git
cd Scrapper
python -m venv .venv
# Windows: .venv\Scripts\activate
# macOS/Linux: source .venv/bin/activate
pip install -r requirements.txt
python scraper.py

Stack

  • Python 3.10+
  • Selenium
  • ChromeDriver
  • python-docx

Scrape the catalog. Keep the document.

Open source on GitHub. Landing page on Vercel. Browser automation stays local.

Open repository