From 55d803f5df489ff8bc407a3ba118c7af583fcdba Mon Sep 17 00:00:00 2001 From: Gil Desmarais Date: Sat, 14 Mar 2026 22:16:22 +0100 Subject: [PATCH 01/11] Align docs with web app UX flow --- .../docs/AutoGenerationOptional.astro | 6 +- src/content/docs/creating-custom-feeds.mdx | 6 +- src/content/docs/getting-started.mdx | 16 +-- src/content/docs/index.mdx | 108 ++++++------------ .../docs/web-application/getting-started.mdx | 65 ++++++----- .../how-to/use-automatic-feed-generation.mdx | 30 +++-- src/content/docs/web-application/index.mdx | 29 ++--- 7 files changed, 120 insertions(+), 140 deletions(-) diff --git a/src/components/docs/AutoGenerationOptional.astro b/src/components/docs/AutoGenerationOptional.astro index 35039fc9..982682bb 100644 --- a/src/components/docs/AutoGenerationOptional.astro +++ b/src/components/docs/AutoGenerationOptional.astro @@ -2,7 +2,7 @@ import { Aside } from "@astrojs/starlight/components"; --- - --- diff --git a/src/content/docs/getting-started.mdx b/src/content/docs/getting-started.mdx index f08de5ba..e0c299b0 100644 --- a/src/content/docs/getting-started.mdx +++ b/src/content/docs/getting-started.mdx @@ -1,6 +1,6 @@ --- title: "Getting Started" -description: "Learn how to get RSS feeds from any website. Start with existing feeds or create your own in minutes." +description: "Start html2rss-web locally, verify the web interface, generate your first feed URL, and decide when to move to custom configs." sidebar: order: 1 --- @@ -14,12 +14,14 @@ If you want the recommended path, go to [Run html2rss-web with Docker](/web-appl That guide is the canonical setup flow for: - running `html2rss-web` locally -- confirming your first successful feed -- deciding when to use included feeds, automatic generation, or custom configs +- confirming the interface is working +- generating a first feed URL +- deciding when to use automatic generation or custom configs ## Quick Shortcuts -- **[Run html2rss-web with Docker](/web-application/getting-started)** - Recommended first step -- **[Browse working feed examples](/feed-directory/)** - See what success looks like -- **[Create Custom Feeds](/creating-custom-feeds)** - Write configs when you need more control -- **[Troubleshooting Guide](/troubleshooting/troubleshooting)** - Fix startup or extraction problems +- **[Run html2rss-web with Docker](/web-application/getting-started)**: recommended first step +- **[Use automatic feed generation](/web-application/how-to/use-automatic-feed-generation/)**: create a feed directly from a page URL +- **[Browse working feed examples](/feed-directory/)**: see what successful outputs look like +- **[Create Custom Feeds](/creating-custom-feeds)**: write configs when you need more control +- **[Troubleshooting Guide](/troubleshooting/troubleshooting)**: fix startup or extraction problems diff --git a/src/content/docs/index.mdx b/src/content/docs/index.mdx index 349bccae..75ab9d0e 100644 --- a/src/content/docs/index.mdx +++ b/src/content/docs/index.mdx @@ -1,101 +1,69 @@ --- -title: "Turn Any Website Into an RSS Feed - Never Miss Updates Again" -description: "Create RSS feeds from any website - no coding required. Turn blogs, news sites, and forums into RSS feeds you can follow in your favorite reader. Free, open source, and easy to use." +title: "Turn Any Website Into an RSS Feed" +description: "Run html2rss-web with Docker, open the web interface, generate stable feed URLs, and move to custom configs only when you need more control." --- -Run `html2rss-web` with Docker, start with included feeds, and add custom configs only when you need more control. +Run `html2rss-web` with Docker, open the web interface, and generate stable feed URLs from pages you want to follow. -## πŸš€ Get Started in 30 Seconds +## Start Here -**Start here:** [Run html2rss-web with Docker](/web-application/getting-started) | [Browse working feed examples](/feed-directory/) +**Recommended path:** [Run html2rss-web with Docker](/web-application/getting-started) -Need more control? [Write a custom feed config](/creating-custom-feeds) +That guide is the canonical onboarding flow for: ---- +- starting a local instance +- verifying the web interface +- generating a first feed URL +- deciding when to use automatic generation or custom configs ## How It Works 1. **Run your own local instance** with Docker -2. **Use included feeds or add your own** website targets -3. **Subscribe from your RSS reader** using stable feed URLs - ---- - -## Why RSS Still Matters Today - -**Real examples of what you can do:** - -- Follow your favorite blogs without social media algorithms -- Get notified when your local news site posts about your neighborhood -- Track job postings from multiple company websites -- Monitor product updates from software vendors -- Follow academic papers from your field - -**RSS vs Social Media:** - -- βœ… **No algorithms** deciding what you see -- βœ… **No ads** or sponsored content -- βœ… **Works with any feed reader** you choose -- βœ… **Your data stays private** -- βœ… **Never miss updates** - automatic notifications -- βœ… **Save time** - no more manual checking - ---- +2. **Open the web interface** and paste a page URL +3. **Copy the feed URL into your reader** ## What is html2rss? -html2rss is a toolkit for turning websites into RSS feeds. Think of it as a translator that converts website content into a format your feed reader can understand. +html2rss is a toolkit for turning websites into feeds. -**Most people should start with the web application:** +Most people should start with the web application: -- **🌐 html2rss-web** - The easiest way to run your own feed server with Docker -- **βš™οΈ html2rss gem** - The underlying engine, CLI, and developer interface +- **`html2rss-web`**: the self-hosted web interface and feed server +- **`html2rss` gem**: the Ruby engine, CLI, and lower-level config workflow ---- - -## 🎯 Choose Your Path +## Choose Your Path ### I want a working instance first -1. **[Run html2rss-web with Docker](/web-application/getting-started)** - Recommended starting path -2. **[Browse working feed examples](/feed-directory/)** - See what success looks like -3. **[Use the included configs](/web-application/how-to/use-included-configs/)** - Start with ready-made feeds +1. **[Run html2rss-web with Docker](/web-application/getting-started)**: recommended starting path +2. **[Use automatic feed generation](/web-application/how-to/use-automatic-feed-generation/)**: create a feed directly from a page URL +3. **[Browse working feed examples](/feed-directory/)**: see what working outputs look like ### I need more control -1. **[Creating Custom Feeds](/creating-custom-feeds)** - Write and test your own configs -2. **[Selectors Reference](/ruby-gem/reference/selectors/)** - Learn the matching rules -3. **[Strategy Reference](/ruby-gem/reference/strategy/)** - Use `browserless` for JS-heavy sites +1. **[Creating Custom Feeds](/creating-custom-feeds)**: write and test your own configs +2. **[Selectors Reference](/ruby-gem/reference/selectors/)**: learn the matching rules +3. **[Strategy Reference](/ruby-gem/reference/strategy/)**: decide when `browserless` is justified ### I'm building or integrating -1. **[Ruby Gem Reference](/ruby-gem/)** - Full API documentation -2. **[Advanced Features](/ruby-gem/how-to/advanced-features/)** - Custom HTTP requests, etc. -3. **[Contribute to Core](/get-involved/contributing/)** - Help improve the engine - ---- - -## 🌟 What People Are Using html2rss For - -- **News & Blogs:** Follow your favorite writers without social media -- **Job Hunting:** Track job postings from multiple company sites -- **Product Updates:** Get notified when software you use gets updated -- **Academic Research:** Follow new papers in your field -- **Local News:** Stay updated on your neighborhood and city -- **Hobby Communities:** Follow forums and communities you care about - -[Browse all examples in our Feed Directory β†’](/feed-directory/) - ---- - -## πŸ”§ Common Issues? +1. **[Ruby Gem Reference](/ruby-gem/)**: full API documentation +2. **[Advanced Features](/ruby-gem/how-to/advanced-features/)**: custom HTTP requests and advanced extraction +3. **[Contribute to Core](/get-involved/contributing/)**: help improve the engine -**Start with Docker, not a public instance.** That gives you the most reliable path and the newest integrated behavior. +## What People Use It For -**Feed not working?** Check our [troubleshooting guide](/troubleshooting/troubleshooting) +- follow blogs and news sites without social media algorithms +- track product updates and release notes +- monitor job postings from company websites +- subscribe to forums and communities that do not publish feeds +- follow local news without repeated manual checking -**Need custom control?** Continue to [Creating Custom Feeds](/creating-custom-feeds) +## Practical Notes -**Need help?** Join our [community discussions](https://github.com/orgs/html2rss/discussions) +- Start with Docker, not a public instance. +- Use the web interface to verify the deployment first. +- Use automatic generation for the first pass. +- Move to custom configs when you need a stable, reviewable setup. -**Found a bug?** [Report it on GitHub](https://github.com/html2rss/html2rss/issues) +**Need help?** Continue to the [troubleshooting guide](/troubleshooting/troubleshooting) or join [GitHub Discussions](https://github.com/orgs/html2rss/discussions). diff --git a/src/content/docs/web-application/getting-started.mdx b/src/content/docs/web-application/getting-started.mdx index 12ac3428..a4a02280 100644 --- a/src/content/docs/web-application/getting-started.mdx +++ b/src/content/docs/web-application/getting-started.mdx @@ -1,6 +1,6 @@ --- title: "Getting Started" -description: "Learn how to use html2rss-web to create RSS feeds from any website. Step-by-step guide for beginners with no technical knowledge required." +description: "Run html2rss-web locally with Docker, verify the interface, and generate your first feed URL." sidebar: order: 2 --- @@ -8,15 +8,16 @@ sidebar: import AutoGenerationOptional from "../../../components/docs/AutoGenerationOptional.astro"; import MinimalDockerCompose from "../../../components/docs/MinimalDockerCompose.astro"; -Ready to create RSS feeds? Run `html2rss-web` locally with Docker, verify the interface, then use included feeds before you move on to auto-generation or custom configs. +Run `html2rss-web` locally with Docker, open the web interface, and verify that you can generate a working feed URL. ## What You Will Have When This Works After this guide, you should have: - `html2rss-web` running at `http://localhost:3000` -- a feed list loaded from `feeds.yml` -- a clear path to either automatic generation or custom YAML configs +- the web interface loading correctly +- a first feed URL you can copy into your reader +- a clear path to either custom configs or more advanced setup ## Installation Guide @@ -24,16 +25,14 @@ This guide walks you through a local Docker setup that gives you the most reliab ### What You'll Need -- **Docker** - A tool that makes installation simple (like an app store for server software) -- **About 10 minutes** - The whole process is quick and automated +- **Docker** +- **About 10 minutes** -**Don't have Docker?** [Install it first](https://docs.docker.com/get-started/) - it's free and works on all major operating systems. +If you do not already have Docker, [install it first](https://docs.docker.com/get-started/). ### Step 1: Create a Folder -Create a new folder on your computer to store html2rss-web files: - -**Create a new folder** on your computer and name it "html2rss-web". You can do this through your file manager or terminal: +Create a new folder for `html2rss-web`: ```bash mkdir html2rss-web @@ -42,20 +41,18 @@ cd html2rss-web ### Step 2: Create a Minimal Configuration File -Create a file called `docker-compose.yml` in your new folder. Start with the minimal local stack: +Create a file called `docker-compose.yml` in that folder and start with the minimal local stack: -Add update automation such as Watchtower later, after the first run works. - -### Step 3: Download the Feed List +Add update automation later, after the first run works. -html2rss-web needs a list of feeds to work with. Download our pre-made list: +### Step 3: Download `feeds.yml` -**Download the feeds.yml file:** +Download the default `feeds.yml` file: -- **Using your browser:** Right-click [this link](https://raw.githubusercontent.com/html2rss/html2rss-web/master/config/feeds.yml) β†’ Save As β†’ Name it "feeds.yml" β†’ Save in your html2rss-web folder -- **Using terminal:** Open Terminal in your html2rss-web folder and run: +- **Browser:** Right-click [this link](https://raw.githubusercontent.com/html2rss/html2rss-web/master/config/feeds.yml) and save it as `feeds.yml` +- **Terminal:** ```bash curl https://raw.githubusercontent.com/html2rss/html2rss-web/master/config/feeds.yml -o feeds.yml @@ -63,7 +60,7 @@ curl https://raw.githubusercontent.com/html2rss/html2rss-web/master/config/feeds ### Step 4: Start html2rss-web -Open a terminal in your html2rss-web folder and run: +Run: ```bash docker compose up -d @@ -73,27 +70,29 @@ docker compose up -d At this point, `html2rss-web` should be running. -1. Open your web browser -2. Go to `http://localhost:3000` -3. You should see the html2rss-web interface with a list of available feeds +1. Open `http://localhost:3000` +2. Confirm the web interface loads +3. Paste a page URL into `Create a feed` +4. Generate a feed URL +5. Copy that feed URL into your reader or open it directly -If you see the interface, the setup worked. +If that works, the deployment, interface, and feed-generation path are working together. ## What To Do First -1. Open one included feed from the list -2. Copy the RSS URL into your reader -3. Confirm you can subscribe successfully +Start with the web interface itself: -That proves the web app, feed config file, and request pipeline are all working together. +1. create one feed from a known page URL +2. open or copy the generated feed URL +3. confirm your reader can subscribe successfully - +That proves the core path before you invest in deeper customization. ---- + ## Next Steps -1. **[Use the included configs](/web-application/how-to/use-included-configs/)** - Start with working feeds -2. **[Use automatic feed generation](/web-application/how-to/use-automatic-feed-generation/)** - Enable the quick-generate workflow -3. **[Create Custom Feeds](/creating-custom-feeds)** - Write and test your own configs -4. **[Need help?](/troubleshooting/troubleshooting)** - Troubleshoot startup and extraction problems +1. **[Use automatic feed generation](/web-application/how-to/use-automatic-feed-generation/)**: understand the web UI creation path in more detail +2. **[Create Custom Feeds](/creating-custom-feeds)**: write your own configs when you need reviewable extraction rules +3. **[Browse working feed examples](/feed-directory/)**: compare outputs and existing configs +4. **[Need help?](/troubleshooting/troubleshooting)**: troubleshoot startup and extraction problems diff --git a/src/content/docs/web-application/how-to/use-automatic-feed-generation.mdx b/src/content/docs/web-application/how-to/use-automatic-feed-generation.mdx index e19c379e..97c3e099 100644 --- a/src/content/docs/web-application/how-to/use-automatic-feed-generation.mdx +++ b/src/content/docs/web-application/how-to/use-automatic-feed-generation.mdx @@ -1,15 +1,15 @@ --- title: "Use automatic feed generation" -description: "This feature lets you create RSS feeds automatically - just enter a website URL and html2rss-web figures out the rest!" +description: "Enable the web UI flow that generates a feed directly from a page URL." --- -This feature lets you create RSS feeds automatically - just enter a website URL and html2rss-web figures out the rest! +Automatic feed generation is the direct web-interface workflow: paste a page URL, create a feed, then copy the generated feed URL. > **Note:** This feature is disabled by default for security reasons. ## How to Enable It -1. **Edit your `docker-compose.yml` file** and uncomment these lines: +Edit your `docker-compose.yml` and enable the automatic generation environment variables: ```yaml environment: @@ -19,7 +19,7 @@ environment: AUTO_SOURCE_ALLOWED_ORIGINS: 127.0.0.1:3000 ``` -2. **Restart html2rss-web:** +Then restart: ```bash docker compose down @@ -28,16 +28,26 @@ docker compose up -d ## How to Use It -1. **Open the auto-source page:** Go to `http://localhost:3000/auto_source/` -2. **Enter your credentials** (the username and password you set above) -3. **Enter a website URL** and click "Generate" -4. **Get your RSS feed!** html2rss-web will create a feed automatically +1. Open your instance at `http://localhost:3000` +2. Paste a page URL into `Create a feed` +3. Submit the form +4. If access is required, provide the configured access token +5. Copy the generated feed URL or open it directly -**That's it!** No configuration files needed - html2rss-web does all the work for you. +## What Success Looks Like + +When the flow works, you should see: + +- a generated feed URL +- a copy action +- an open-feed action +- a preview of recent entries when available + +That is enough to confirm the endpoint is live. ## When to Stop and Switch -Automatic feed generation is a fast first try, not the final answer for every site. +Automatic feed generation is the fast first pass, not the final answer for every site. Move on to [Creating Custom Feeds](/creating-custom-feeds) when: diff --git a/src/content/docs/web-application/index.mdx b/src/content/docs/web-application/index.mdx index c5826bb7..220d60e8 100644 --- a/src/content/docs/web-application/index.mdx +++ b/src/content/docs/web-application/index.mdx @@ -1,34 +1,35 @@ --- title: "Web Application" -description: "html2rss-web is an easy-to-use web application that creates RSS feeds from any website. No coding required - just point, click, and get your RSS feed." +description: "html2rss-web is the self-hosted web interface for generating stable feed URLs from web pages." sidebar: label: "Overview" order: 1 --- -`html2rss-web` is the recommended way to get started. Run it locally with Docker, use included feeds first, and move to automatic generation or custom configs only when you need more control. +`html2rss-web` is the recommended way to get started. Run it locally with Docker, open the interface, generate a feed URL, and move to custom configs only when you need more control. ## Get Started Start with **[Getting Started](/web-application/getting-started)** to: - run your own local instance -- confirm a first successful feed +- verify the web interface +- generate a first feed URL - choose the right next step for your site -## Key Features +## What The Web App Gives You -- **Stable URLs:** Provides stable URLs for feeds served from your own instance -- **Included Feeds:** Start with many working configs out of the box -- **Automatic Generation:** Try new sites quickly when the feature is enabled -- **Custom Feeds:** Create and refine your own configs when you need more control -- **Caching:** Handles request caching and sets HTTP headers +- **Stable feed URLs:** generated from your own deployment +- **Web interface:** paste a page URL and create a feed +- **Optional access control:** unlock custom generation with an access token when configured +- **Config-based extension path:** move to custom feeds when you need reviewable rules +- **Caching and HTTP handling:** shipped as part of the deployment -The functionality of scraping websites and building the RSS feeds is provided by the Ruby gem [`html2rss`](https://github.com/html2rss/html2rss). +The scraping and feed-building engine is provided by the Ruby gem [`html2rss`](https://github.com/html2rss/html2rss). ## Recommended Flow -1. **[Getting Started](/web-application/getting-started)** - Run the app locally -2. **[Use the included configs](/web-application/how-to/use-included-configs/)** - Confirm a working feed first -3. **[Use automatic feed generation](/web-application/how-to/use-automatic-feed-generation/)** - Try unsupported sites quickly -4. **[Create Custom Feeds](/creating-custom-feeds)** - Build a stable custom setup when needed +1. **[Getting Started](/web-application/getting-started)**: run the app locally +2. **[Use automatic feed generation](/web-application/how-to/use-automatic-feed-generation/)**: create a feed from a page URL +3. **[Browse working feed examples](/feed-directory/)**: compare against existing outputs +4. **[Create Custom Feeds](/creating-custom-feeds)**: build a stable custom setup when needed From 3fbbaf8b922fc76993875c15b2b68ba5107f9fc3 Mon Sep 17 00:00:00 2001 From: Gil Desmarais Date: Sun, 15 Mar 2026 12:00:03 +0100 Subject: [PATCH 02/11] Update docs for renamed Docker Hub image --- examples/deployment/docker-compose.yml | 39 +++++++ .../docs/DockerComposeSnippet.astro | 110 ++++++++++++++++++ .../docs/MinimalDockerCompose.astro | 32 +---- .../web-application/how-to/deployment.mdx | 70 +---------- .../reference/versioning-and-releases.mdx | 4 +- src/data/docker.ts | 6 + 6 files changed, 166 insertions(+), 95 deletions(-) create mode 100644 examples/deployment/docker-compose.yml create mode 100644 src/components/docs/DockerComposeSnippet.astro create mode 100644 src/data/docker.ts diff --git a/examples/deployment/docker-compose.yml b/examples/deployment/docker-compose.yml new file mode 100644 index 00000000..5b98fa66 --- /dev/null +++ b/examples/deployment/docker-compose.yml @@ -0,0 +1,39 @@ +services: + html2rss: + image: html2rss/web:latest + env_file: .env + + caddy: + image: caddy:2-alpine + depends_on: + - html2rss + command: + - caddy + - reverse-proxy + - --from + - ${CADDY_HOST} + - --to + - html2rss:3000 + ports: + - "80:80" + - "443:443" + volumes: + - caddy_data:/data + + watchtower: + image: containrrr/watchtower + depends_on: + - html2rss + - caddy + command: + - --cleanup + - --interval + - "300" + - html2rss + - caddy + volumes: + - /var/run/docker.sock:/var/run/docker.sock:ro + restart: unless-stopped + +volumes: + caddy_data: diff --git a/src/components/docs/DockerComposeSnippet.astro b/src/components/docs/DockerComposeSnippet.astro new file mode 100644 index 00000000..16c9ef11 --- /dev/null +++ b/src/components/docs/DockerComposeSnippet.astro @@ -0,0 +1,110 @@ +--- +import Code from "astro/components/Code.astro"; +import { + browserlessImage, + caddyImage, + watchtowerImage, + webImage, +} from "../../data/docker"; + +interface Props { + variant: + | "minimal" + | "productionCaddy" + | "secure" + | "watchtower" + | "resourceGuardrails"; +} + +const { variant } = Astro.props; + +const snippets: Record = { + minimal: `services: + html2rss-web: + image: ${webImage} + restart: unless-stopped + ports: + - "127.0.0.1:3000:3000" + volumes: + - type: bind + source: ./feeds.yml + target: /app/config/feeds.yml + read_only: true + environment: + RACK_ENV: production + HEALTH_CHECK_USERNAME: health + HEALTH_CHECK_PASSWORD: CHANGE_THIS_PASSWORD_BEFORE_USE + BROWSERLESS_IO_WEBSOCKET_URL: ws://browserless:3001 + BROWSERLESS_IO_API_TOKEN: 6R0W53R135510 + + browserless: + image: "${browserlessImage}" + restart: unless-stopped + ports: + - "127.0.0.1:3001:3001" + environment: + PORT: 3001 + CONCURRENT: 10 + TOKEN: 6R0W53R135510`, + productionCaddy: `services: + caddy: + image: ${caddyImage} + ports: + - "80:80" + - "443:443" + volumes: + - caddy_data:/data + command: + - caddy + - reverse-proxy + - --from + - \${CADDY_HOST} + - --to + - html2rss:3000 + html2rss: + image: ${webImage} + env_file: .env + +volumes: + caddy_data:`, + secure: `services: + html2rss: + image: ${webImage} + environment: + RACK_ENV: production + LOG_LEVEL: warn + HEALTH_CHECK_USERNAME: your-secure-username + HEALTH_CHECK_PASSWORD: your-very-secure-password + BASE_URL: https://yourdomain.com`, + watchtower: `services: + watchtower: + image: ${watchtowerImage} + depends_on: + - html2rss + - caddy + command: + - --cleanup + - --interval + - "300" + - html2rss + - caddy + volumes: + - /var/run/docker.sock:/var/run/docker.sock:ro + restart: unless-stopped`, + resourceGuardrails: `services: + html2rss: + image: ${webImage} + deploy: + resources: + limits: + memory: 512M + cpus: "0.5" + reservations: + memory: 256M + cpus: "0.25"`, +}; + +const code = snippets[variant]; +--- + + diff --git a/src/components/docs/MinimalDockerCompose.astro b/src/components/docs/MinimalDockerCompose.astro index ebc0001c..9eb6aeb0 100644 --- a/src/components/docs/MinimalDockerCompose.astro +++ b/src/components/docs/MinimalDockerCompose.astro @@ -1,33 +1,5 @@ --- -import Code from "astro/components/Code.astro"; - -const code = `services: - html2rss-web: - image: gilcreator/html2rss-web - restart: unless-stopped - ports: - - "127.0.0.1:3000:3000" - volumes: - - type: bind - source: ./feeds.yml - target: /app/config/feeds.yml - read_only: true - environment: - RACK_ENV: production - HEALTH_CHECK_USERNAME: health - HEALTH_CHECK_PASSWORD: CHANGE_THIS_PASSWORD_BEFORE_USE - BROWSERLESS_IO_WEBSOCKET_URL: ws://browserless:3001 - BROWSERLESS_IO_API_TOKEN: 6R0W53R135510 - - browserless: - image: "ghcr.io/browserless/chromium" - restart: unless-stopped - ports: - - "127.0.0.1:3001:3001" - environment: - PORT: 3001 - CONCURRENT: 10 - TOKEN: 6R0W53R135510`; +import DockerComposeSnippet from "./DockerComposeSnippet.astro"; --- - + diff --git a/src/content/docs/web-application/how-to/deployment.mdx b/src/content/docs/web-application/how-to/deployment.mdx index 20116856..df7c07dd 100644 --- a/src/content/docs/web-application/how-to/deployment.mdx +++ b/src/content/docs/web-application/how-to/deployment.mdx @@ -3,6 +3,8 @@ title: "Deployment & Production" description: "Deploy html2rss-web to production with Docker. Learn best practices for hosting public instances with security, monitoring, and reliability." --- +import DockerComposeSnippet from "../../../../components/docs/DockerComposeSnippet.astro"; + html2rss-web ships on Docker Hub, so you can launch it wherever Docker runs. Start with the official [`docker-compose.yml`](https://github.com/html2rss/html2rss-web/blob/master/docker-compose.yml) from the [Installation Guide](/web-application/getting-started) as your baseline. If you have not yet created a local instance, complete the [Getting Started guide](/web-application/getting-started) first. It walks through the one-time project directory setup, downloading the reference compose file, and confirming the application locallyβ€”steps we will build upon here. @@ -25,29 +27,7 @@ A reverse proxy accepts public HTTPS traffic, terminates TLS, and forwards reque Caddy handles certificates and redirects with almost no configuration. -```yaml -services: - caddy: - image: caddy:2-alpine - ports: - - "80:80" - - "443:443" - volumes: - - caddy_data:/data - command: - - caddy - - reverse-proxy - - --from - - ${CADDY_HOST} - - --to - - html2rss:3000 - html2rss: - image: gilcreator/html2rss-web:latest - env_file: .env - -volumes: - caddy_data: -``` + - Create a `.env` file beside your compose file with the following variables: @@ -79,17 +59,7 @@ Harden the application before inviting other users: - Prefer environment files (`.env`) stored outside version control for secrets - Keep admin-only routes behind basic auth or IP restrictions in your proxy -```yaml -services: - html2rss: - image: gilcreator/html2rss-web:latest - environment: - RACK_ENV: production - LOG_LEVEL: warn - HEALTH_CHECK_USERNAME: your-secure-username - HEALTH_CHECK_PASSWORD: your-very-secure-password - BASE_URL: https://yourdomain.com -``` + Store these variables in a `.env` file and reference it with `env_file:` as demonstrated in the Caddy example. @@ -104,41 +74,13 @@ Keep the instance healthy once it is in production: ### Auto-update with Watchtower -```yaml -services: - watchtower: - image: containrrr/watchtower - depends_on: - - html2rss - - caddy - command: - - --cleanup - - --interval - - "300" - - html2rss - - caddy - volumes: - - /var/run/docker.sock:/var/run/docker.sock - restart: unless-stopped -``` + Check `docker compose logs watchtower` occasionally to confirm updates are applied. ### Resource Guardrails -```yaml -services: - html2rss: - image: gilcreator/html2rss-web:latest - deploy: - resources: - limits: - memory: 512M - cpus: "0.5" - reservations: - memory: 256M - cpus: "0.25" -``` + Adjust the limits to match your host capacity. Increase memory if you process many large feeds. diff --git a/src/content/docs/web-application/reference/versioning-and-releases.mdx b/src/content/docs/web-application/reference/versioning-and-releases.mdx index 63de78d2..903adff8 100644 --- a/src/content/docs/web-application/reference/versioning-and-releases.mdx +++ b/src/content/docs/web-application/reference/versioning-and-releases.mdx @@ -3,9 +3,11 @@ title: Versioning and releases description: Learn about versioning and release strategy for html2rss-web --- +import { dockerHubRepository, dockerHubUrl } from "../../../../data/docker"; + This web application is distributed in a [rolling release](https://en.wikipedia.org/wiki/Rolling_release) fashion from the `master` branch. -For the latest commit passing GitHub CI/CD on the master branch, an updated Docker image will be pushed to [Docker Hub: `gilcreator/html2rss-web`](https://hub.docker.com/r/gilcreator/html2rss-web). +For the latest commit passing GitHub CI/CD on the master branch, an updated Docker image will be pushed to Docker Hub: {dockerHubRepository}. The [SBOM](https://en.wikipedia.org/wiki/Software_supply_chain) is embedded in the Docker image. GitHub's @dependabot is enabled for dependency updates and they are automatically merged to the `master` branch when the CI gives the green light. diff --git a/src/data/docker.ts b/src/data/docker.ts new file mode 100644 index 00000000..922948d5 --- /dev/null +++ b/src/data/docker.ts @@ -0,0 +1,6 @@ +export const dockerHubRepository = "html2rss/web"; +export const dockerHubUrl = `https://hub.docker.com/r/${dockerHubRepository}`; +export const webImage = `${dockerHubRepository}:latest`; +export const browserlessImage = "ghcr.io/browserless/chromium"; +export const caddyImage = "caddy:2-alpine"; +export const watchtowerImage = "containrrr/watchtower"; From ab7296c5646070788abb509f4edb01e1960f3f5d Mon Sep 17 00:00:00 2001 From: Gil Desmarais Date: Wed, 18 Mar 2026 11:54:15 +0100 Subject: [PATCH 03/11] docs: add wordpress-api and update gem docs --- src/content/docs/creating-custom-feeds.mdx | 15 +++ src/content/docs/getting-started.mdx | 16 ++++ .../ruby-gem/how-to/advanced-features.mdx | 10 +- .../docs/ruby-gem/reference/auto-source.mdx | 13 ++- .../docs/ruby-gem/reference/selectors.mdx | 6 +- .../docs/ruby-gem/reference/wordpress-api.mdx | 91 +++++++++++++++++++ 6 files changed, 141 insertions(+), 10 deletions(-) create mode 100644 src/content/docs/ruby-gem/reference/wordpress-api.mdx diff --git a/src/content/docs/creating-custom-feeds.mdx b/src/content/docs/creating-custom-feeds.mdx index 203bc9cf..b20aecf5 100644 --- a/src/content/docs/creating-custom-feeds.mdx +++ b/src/content/docs/creating-custom-feeds.mdx @@ -160,6 +160,21 @@ html2rss supports many configuration options: 4. **Check the output:** Make sure all items have titles, links, and descriptions +### Useful CLI flags when a site is difficult + +Some sites need a little more request budget than the defaults. + +- Use `--max-redirects` when the site bounces through several canonicalization or tracking redirects before the real page loads. +- Use `--max-requests` when your config needs more than one request, for example pagination or other follow-up fetches. + +```bash +html2rss feed your-config.yml --max-redirects 10 +html2rss feed your-config.yml --max-requests 5 +html2rss auto https://example.com/blog --max-redirects 10 --max-requests 5 +``` + +Keep these values as low as possible. If a site only needs one extra redirect, prefer `--max-redirects 4` over a much larger number. + ## Add It To html2rss-web Once the config works locally, add it to your `feeds.yml` or shared config repository and restart your diff --git a/src/content/docs/getting-started.mdx b/src/content/docs/getting-started.mdx index e0c299b0..30ea4175 100644 --- a/src/content/docs/getting-started.mdx +++ b/src/content/docs/getting-started.mdx @@ -25,3 +25,19 @@ That guide is the canonical setup flow for: - **[Browse working feed examples](/feed-directory/)**: see what successful outputs look like - **[Create Custom Feeds](/creating-custom-feeds)**: write configs when you need more control - **[Troubleshooting Guide](/troubleshooting/troubleshooting)**: fix startup or extraction problems + +## Using the Ruby CLI + +If you are working directly with the gem instead of `html2rss-web`, start with: + +```bash +html2rss auto https://example.com/blog +``` + +If the target site is unusually redirect-heavy or needs extra follow-up requests, the CLI also supports: + +```bash +html2rss auto https://example.com/blog --max-redirects 10 --max-requests 5 +``` + +For config-driven runs, the same flags are available on `html2rss feed`. diff --git a/src/content/docs/ruby-gem/how-to/advanced-features.mdx b/src/content/docs/ruby-gem/how-to/advanced-features.mdx index 703bd9e9..cf052a64 100644 --- a/src/content/docs/ruby-gem/how-to/advanced-features.mdx +++ b/src/content/docs/ruby-gem/how-to/advanced-features.mdx @@ -7,13 +7,13 @@ This guide covers advanced features and performance optimizations for html2rss. ## Parallel Processing -html2rss uses parallel processing to improve performance when scraping multiple items. This happens automatically and doesn't require any configuration. +html2rss uses parallel processing in auto-source discovery to improve performance when multiple scrapers inspect the same page. This happens automatically and doesn't require any configuration. ### How It Works -- **Auto-source scraping:** Multiple scrapers run in parallel to analyze the page -- **Item processing:** Each scraped item is processed in parallel -- **Performance benefit:** Significantly faster when dealing with many items +- **Auto-source scraping:** Multiple scrapers run in parallel to analyze the same response body +- **Selectors and pagination:** Selector extraction and `rel="next"` pagination stay sequential and share the same request budget +- **Performance benefit:** Faster auto-discovery without changing selector semantics ### Performance Tips @@ -75,6 +75,8 @@ selectors: extractor: "href" ``` +When you use the Browserless strategy, Chromium rejects transport-level headers such as `Host`, `Connection`, `Content-Length`, and `Transfer-Encoding`. html2rss filters those headers before navigation and logs the filtered header names at `info` level. + ## Monitoring and Debugging ### Enable Debug Logging diff --git a/src/content/docs/ruby-gem/reference/auto-source.mdx b/src/content/docs/ruby-gem/reference/auto-source.mdx index 33454232..82e92df0 100644 --- a/src/content/docs/ruby-gem/reference/auto-source.mdx +++ b/src/content/docs/ruby-gem/reference/auto-source.mdx @@ -17,16 +17,19 @@ auto_source: {} `auto_source` uses the following strategies to find content: -1. **`schema`:** Parses ` diff --git a/src/components/feed-directory.js b/src/components/feed-directory.js index d7d42f07..d690ffd6 100644 --- a/src/components/feed-directory.js +++ b/src/components/feed-directory.js @@ -1,7 +1,5 @@ // Feed Directory JavaScript functionality -// Simple, focused functions for maintainability -// Simple debounce helper function debounce(func, wait) { let timeout; return function executedFunction(...args) { @@ -14,7 +12,45 @@ function debounce(func, wait) { }; } -// Simple fuzzy search +function getDefaultInstanceUrl() { + return atob('aHR0cHM6Ly8xLmgyci53b3JrZXJzLmRldi8='); +} + +function getHashParams() { + const hash = window.location.hash || ''; + const normalizedHash = hash.startsWith('#!') ? hash.slice(2) : hash.startsWith('#') ? hash.slice(1) : hash; + return new URLSearchParams(normalizedHash); +} + +function readInstanceUrlFromHash(defaultInstanceUrl) { + const candidate = getHashParams().get('url'); + if (!candidate) return defaultInstanceUrl; + + try { + const parsedUrl = new URL(candidate); + parsedUrl.search = ''; + parsedUrl.hash = ''; + return parsedUrl.toString(); + } catch { + return defaultInstanceUrl; + } +} + +function writeInstanceUrlToHash(instanceUrl, defaultInstanceUrl) { + const params = getHashParams(); + if (instanceUrl && instanceUrl !== defaultInstanceUrl) { + params.set('url', instanceUrl); + } else { + params.delete('url'); + } + + const nextHash = params.toString(); + const nextUrl = nextHash + ? `${window.location.pathname}${window.location.search}#!${nextHash}` + : `${window.location.pathname}${window.location.search}`; + window.history.replaceState({}, '', nextUrl); +} + function fuzzyMatch(text, query) { if (!query) return true; const lowerText = text.toLowerCase(); @@ -28,150 +64,284 @@ function fuzzyMatch(text, query) { return queryIndex === lowerQuery.length; } -// Search functionality +function formatInstanceLabel(instanceUrl) { + try { + const parsedUrl = new URL(instanceUrl); + return parsedUrl.host + parsedUrl.pathname.replace(/\/$/, ''); + } catch { + return instanceUrl; + } +} + +function normalizeInstanceUrl(value) { + const trimmed = value.trim(); + if (!trimmed) return null; + + try { + const parsedUrl = new URL(trimmed); + parsedUrl.search = ''; + parsedUrl.hash = ''; + return parsedUrl.toString(); + } catch { + return null; + } +} + +function updateInstanceSummary(instanceUrl) { + const host = document.querySelector('[data-instance-host]'); + if (host) { + host.textContent = formatInstanceLabel(instanceUrl); + } +} + +function setInstanceFeedback(message, state) { + const feedback = document.querySelector('[data-instance-feedback]'); + if (!feedback) return; + feedback.textContent = message; + feedback.dataset.state = state; +} + +function createUpdateFeedUrlsFunction() { + return function updateFeedUrls(instanceUrl) { + document.querySelectorAll('[data-feed-url]').forEach((link) => { + const item = link.closest('[data-domain]'); + if (!item) return; + + const domain = item.dataset.domain; + const name = item.dataset.name; + if (!domain || !name) return; + + const params = {}; + item.querySelectorAll('[data-param-key]').forEach((input) => { + if (input.value) params[input.dataset.paramKey] = input.value; + }); + + let url = instanceUrl.endsWith('/') ? instanceUrl : `${instanceUrl}/`; + url += `${domain}/${name}.rss`; + + const queryParams = new URLSearchParams(); + Object.entries(params).forEach(([key, value]) => queryParams.append(key, value)); + const queryString = queryParams.toString(); + if (queryString) url += `?${queryString}`; + + link.href = url; + }); + }; +} + +function updateSearchState(feedItems, query) { + let visibleCount = 0; + feedItems.forEach((item) => { + const searchableText = item.dataset.searchable?.toLowerCase() || ''; + const matches = fuzzyMatch(searchableText, query); + item.hidden = !matches; + if (matches) visibleCount++; + }); + + const resultCount = document.querySelector('[data-result-count]'); + const resultLabel = document.querySelector('[data-result-label]'); + const emptyState = document.querySelector('[data-empty-state]'); + const emptyCopy = document.querySelector('[data-empty-copy]'); + const feedList = document.querySelector('[data-feed-list]'); + + if (resultCount) { + resultCount.textContent = String(visibleCount); + } + + if (resultLabel) { + const hasQuery = Boolean(query.trim()); + if (hasQuery) { + resultLabel.textContent = visibleCount === 1 ? 'matching feed' : 'matching feeds'; + } else { + resultLabel.textContent = visibleCount === 1 ? 'ready-to-use feed' : 'ready-to-use feeds'; + } + } + + if (emptyState && emptyCopy && feedList) { + const hasNoResults = visibleCount === 0; + emptyState.hidden = !hasNoResults; + feedList.hidden = hasNoResults; + if (hasNoResults) { + emptyCopy.textContent = query + ? `No configurations found for "${query}". Try another domain or feed name, check the community wiki, or contribute a new configuration.` + : 'Try a different domain or feed name, or contribute a new configuration.'; + } + } +} + function setupSearch(searchInput, feedItems) { if (!searchInput) return; - searchInput.addEventListener( - 'input', - debounce((e) => { - const query = e.target.value.toLowerCase(); - feedItems.forEach((item) => { - const searchableText = item.dataset.searchable?.toLowerCase() || ''; - item.style.display = fuzzyMatch(searchableText, query) ? 'flex' : 'none'; - }); - }, 150) - ); + const applySearch = debounce((query) => { + updateSearchState(feedItems, query.toLowerCase()); + }, 120); + + searchInput.addEventListener('input', (event) => { + applySearch(event.target.value); + }); + + updateSearchState(feedItems, searchInput.value.toLowerCase()); } -// Instance URL updates -function setupInstanceUrlUpdates(instanceInput, defaultInstanceUrl, updateFeedUrls) { - if (!instanceInput) return; +function setupInstanceEditor( + defaultInstanceUrl, + getCurrentInstanceUrl, + setCurrentInstanceUrl, + updateFeedUrls +) { + const toggle = document.querySelector('[data-toggle-instance]'); + const editor = document.getElementById('instance-editor'); + const input = document.getElementById('instance-url-input'); + const apply = document.querySelector('[data-apply-instance]'); + if (!toggle || !editor || !input || !apply) return; - instanceInput.addEventListener( - 'input', - debounce((e) => { - const instanceUrl = e.target.value || defaultInstanceUrl; - updateFeedUrls(instanceUrl); - }, 300) - ); + const setExpanded = (expanded) => { + editor.hidden = !expanded; + toggle.setAttribute('aria-expanded', String(expanded)); + toggle.textContent = expanded ? 'Close' : 'Change'; + }; + + toggle.addEventListener('click', () => { + const nextExpanded = editor.hidden; + setExpanded(nextExpanded); + if (nextExpanded) { + input.value = getCurrentInstanceUrl(); + setInstanceFeedback('Feed links update when you apply a valid instance URL.', 'idle'); + input.focus(); + input.select(); + } + }); + + const applyInstance = () => { + const normalized = normalizeInstanceUrl(input.value); + if (!normalized) { + setInstanceFeedback('Enter a valid URL.', 'error'); + return; + } + + setCurrentInstanceUrl(normalized); + input.value = normalized; + updateInstanceSummary(normalized); + writeInstanceUrlToHash(normalized, defaultInstanceUrl); + updateFeedUrls(normalized); + setExpanded(false); + setInstanceFeedback('Using your custom instance.', 'success'); + }; + + apply.addEventListener('click', applyInstance); + input.addEventListener('keydown', (event) => { + if (event.key === 'Enter') { + event.preventDefault(); + applyInstance(); + } + }); } -// Parameter forms toggle -function setupParameterForms(updateFeedUrls) { - document.querySelectorAll('.configure-button').forEach((button) => { - button.addEventListener('click', (e) => { - const targetId = e.target.closest('button')?.dataset.target; - const form = document.getElementById(targetId); +function setupParameterForms(updateFeedUrls, getCurrentInstanceUrl) { + document.querySelectorAll('[data-target]').forEach((button) => { + button.addEventListener('click', (event) => { + const sourceButton = event.currentTarget; + const form = document.getElementById(sourceButton.dataset.target); if (!form) return; const isExpanded = !form.hidden; form.hidden = isExpanded; - const button = e.target.closest('button'); - if (button) { - button.setAttribute('aria-expanded', !isExpanded); - const span = button.querySelector('span'); - if (span) { - span.textContent = isExpanded ? 'Configure' : 'Hide'; - } + sourceButton.setAttribute('aria-expanded', String(!isExpanded)); + const label = sourceButton.querySelector('span'); + if (label) { + label.textContent = isExpanded ? 'Customize' : 'Close'; } - if (!isExpanded) updateFeedUrls(); + if (!isExpanded) { + updateFeedUrls(getCurrentInstanceUrl()); + } }); }); } -// Close forms function setupCloseForms() { document.querySelectorAll('[data-close-form]').forEach((button) => { - button.addEventListener('click', (e) => { - const form = e.target.closest('.parameter-form'); + button.addEventListener('click', (event) => { + const form = event.currentTarget.closest('.parameter-form'); const toggle = document.querySelector(`[data-target="${form?.id}"]`); if (!form || !toggle) return; form.hidden = true; toggle.setAttribute('aria-expanded', 'false'); - const span = toggle.querySelector('span'); - if (span) { - span.textContent = 'Configure'; + const label = toggle.querySelector('span'); + if (label) { + label.textContent = 'Customize'; } }); }); } -// Parameter input updates -function setupParameterInputs(updateFeedUrls) { +function setupParameterInputs(updateFeedUrls, getCurrentInstanceUrl) { document.querySelectorAll('.form-input').forEach((input) => { input.addEventListener( 'input', - debounce(() => updateFeedUrls(), 200) + debounce(() => updateFeedUrls(getCurrentInstanceUrl()), 180) ); }); } -// Update feed URLs -function createUpdateFeedUrlsFunction() { - return function updateFeedUrls(instanceUrl) { - document.querySelectorAll('[data-feed-url]').forEach((link) => { - const item = link.closest('[data-domain]'); - if (!item) return; - - const domain = item.dataset.domain; - const name = item.dataset.name; - if (!domain || !name) return; - - // Get parameters - const params = {}; - item.querySelectorAll('[data-param-key]').forEach((input) => { - if (input.value) params[input.dataset.paramKey] = input.value; - }); - - // Build URL - let url = instanceUrl.endsWith('/') ? instanceUrl : `${instanceUrl}/`; - url += `${domain}/${name}.rss`; +function setupCopyButtons() { + document.querySelectorAll('[data-copy-feed]').forEach((button) => { + button.addEventListener('click', async () => { + const feedLink = button.closest('[data-domain]')?.querySelector('[data-feed-url]'); + if (!feedLink?.href) return; - const queryParams = new URLSearchParams(); - Object.entries(params).forEach(([key, value]) => queryParams.append(key, value)); - const queryString = queryParams.toString(); - if (queryString) url += `?${queryString}`; - - link.href = url; + try { + await navigator.clipboard.writeText(feedLink.href); + const label = button.querySelector('span'); + button.dataset.copied = 'true'; + if (label) label.textContent = 'Copied'; + window.setTimeout(() => { + button.dataset.copied = 'false'; + if (label) label.textContent = 'Copy link'; + }, 1400); + } catch { + const label = button.querySelector('span'); + if (label) label.textContent = 'Copy failed'; + window.setTimeout(() => { + if (label) label.textContent = 'Copy link'; + }, 1400); + } }); - }; + }); } -// Main initialization function initializeFeedDirectory() { - const defaultInstanceUrl = atob('aHR0cHM6Ly8xLmgyci53b3JrZXJzLmRldi8='); - let instanceUrl = defaultInstanceUrl; + const defaultInstanceUrl = getDefaultInstanceUrl(); + let currentInstanceUrl = readInstanceUrlFromHash(defaultInstanceUrl); const searchInput = document.getElementById('search-input'); - const feedItems = document.querySelectorAll('[data-domain]'); + const feedItems = Array.from(document.querySelectorAll('[data-domain]')); const instanceInput = document.getElementById('instance-url-input'); + const updateFeedUrls = createUpdateFeedUrlsFunction(); + + updateInstanceSummary(currentInstanceUrl); - // Initialize instance URL field if (instanceInput) { - instanceInput.value = defaultInstanceUrl; - instanceUrl = defaultInstanceUrl; + instanceInput.value = currentInstanceUrl; } - // Create update function with current instance URL - const updateFeedUrls = createUpdateFeedUrlsFunction(); - - // Setup all functionality setupSearch(searchInput, feedItems); - setupInstanceUrlUpdates(instanceInput, defaultInstanceUrl, (newUrl) => { - instanceUrl = newUrl; - updateFeedUrls(instanceUrl); - }); - setupParameterForms(() => updateFeedUrls(instanceUrl)); + setupInstanceEditor( + defaultInstanceUrl, + () => currentInstanceUrl, + (nextUrl) => { + currentInstanceUrl = nextUrl; + }, + updateFeedUrls + ); + setupParameterForms(updateFeedUrls, () => currentInstanceUrl); setupCloseForms(); - setupParameterInputs(() => updateFeedUrls(instanceUrl)); + setupParameterInputs(updateFeedUrls, () => currentInstanceUrl); + setupCopyButtons(); - // Initialize feed URLs on page load - updateFeedUrls(instanceUrl); + updateFeedUrls(currentInstanceUrl); } -// Initialize when DOM is ready if (document.readyState === 'loading') { document.addEventListener('DOMContentLoaded', initializeFeedDirectory); } else { diff --git a/src/content/docs/feed-directory/index.mdx b/src/content/docs/feed-directory/index.mdx index e0a3bf94..4b75c0c1 100644 --- a/src/content/docs/feed-directory/index.mdx +++ b/src/content/docs/feed-directory/index.mdx @@ -8,21 +8,15 @@ head: content: noindex --- -This directory contains a list of pre-built configurations to create RSS feeds for various websites. - -## Instance URL - -An Instance URL is the address of a running `html2rss-web` application. You can use a public instance, but we encourage you to host your own. - -[πŸš€ Host Your Own Instance (and share it!)](/web-application/how-to/deployment) +import FeedDirectory from "../../../components/FeedDirectory.astro"; -Find more public instances on the [community-run wiki](https://github.com/html2rss/html2rss-web/wiki/Instances). + --- -import FeedDirectory from "../../../components/FeedDirectory.astro"; +Need a different instance? You can use the built-in default, self-host your own, or find more options on the [community-run wiki](https://github.com/html2rss/html2rss-web/wiki/Instances). - +[πŸš€ Host Your Own Instance (and share it!)](/web-application/how-to/deployment) --- @@ -30,4 +24,4 @@ import FeedDirectory from "../../../components/FeedDirectory.astro"; The feed configurations in this directory are community-driven. If you've created a new feed configuration, we encourage you to share it with the community. -[Contribute on GitHub](https://github.com/html2rss/html2rss-configs) +[Contribute on GitHub](https://github.com/html2rss/html2rss-configs/tree/master/lib/html2rss/configs) From 39d3748af4f3b801273c52e2013844691482a302 Mon Sep 17 00:00:00 2001 From: Gil Desmarais Date: Sat, 21 Mar 2026 13:01:40 +0100 Subject: [PATCH 10/11] Align feed row utility links --- src/components/FeedDirectory.astro | 225 ++++++++++++++--------------- 1 file changed, 106 insertions(+), 119 deletions(-) diff --git a/src/components/FeedDirectory.astro b/src/components/FeedDirectory.astro index 9dc165bf..eed22330 100644 --- a/src/components/FeedDirectory.astro +++ b/src/components/FeedDirectory.astro @@ -15,6 +15,10 @@ const staticFeedUrls = configs.map((config) => ({ ...config, staticFeedUrl: "#", defaultSummary: formatDefaultParameters(config.default_parameters), + sourceSummary: config.channel?.url + ?.replace(/^https?:\/\//, "") + .replace(/^www\./, "") + .replace(/\/$/, ""), })); --- @@ -104,11 +108,6 @@ const staticFeedUrls = configs.map((config) => ({
- - { staticFeedUrls.map((config, index) => (
({ data-searchable={`${config.domain}/${config.name} ${config.channel?.url || ""}`} >
-

- {config.domain} - / - {config.name} -

- - {config.channel?.url && ( - - {config.channel.url} - - )} - - {config.defaultSummary &&

Using defaults: {config.defaultSummary}

} - -
- {Object.keys(config.url_parameters || {}).length > 0 && ( +
+

{config.sourceSummary || `${config.domain}/${config.name}`}

+ +
+ + RSS + +
+
+ +
+ {config.defaultSummary ? ( +

Defaults: {config.defaultSummary}

+ ) : ( + + )} + +
+ {config.channel?.url && ( + + View source + + )} + + {Object.keys(config.url_parameters || {}).length > 0 && ( + + )} + - )} - - - - - Edit - +
-
- - RSS - -
- {Object.keys(config.url_parameters || {}).length > 0 && ( ))} +
+ + Edit + + @@ -270,7 +274,6 @@ const staticFeedUrls = configs.map((config) => ({ .search-block, .instance-block, .empty-state, - .feed-table-head, .feed-row, .parameter-form { padding: var(--fd-padding); @@ -411,17 +414,6 @@ const staticFeedUrls = configs.map((config) => ({ overflow: hidden; } - .feed-table-head { - display: none; - grid-template-columns: minmax(0, 1fr) auto; - gap: 1rem; - border-bottom: 1px solid var(--fd-border); - color: var(--fd-muted); - font-size: var(--sl-text-xs); - letter-spacing: 0.04em; - text-transform: uppercase; - } - .feed-row { display: grid; grid-template-columns: minmax(0, 1fr); @@ -441,54 +433,42 @@ const staticFeedUrls = configs.map((config) => ({ .feed-info { min-width: 0; display: grid; - gap: 0.25rem; + gap: var(--fd-gap); } - .feed-title { - display: flex; - flex-wrap: wrap; + .feed-mainline { + display: grid; + grid-template-columns: minmax(0, 1fr) auto; + gap: 1rem; align-items: baseline; - gap: 0.25rem; - font-size: var(--sl-text-base); - line-height: var(--sl-line-height-headings); - } - - .feed-domain { - color: var(--sl-color-gray-2); - font-weight: 500; } - .feed-separator { - color: var(--sl-color-gray-4); + .feed-subline { + display: grid; + grid-template-columns: minmax(0, 1fr) auto; + gap: 1rem; + align-items: end; } - .feed-name { + .feed-title { + font-size: var(--sl-text-base); + line-height: var(--sl-line-height-headings); color: var(--sl-color-white); font-weight: 600; } - .source-link { - display: block; - overflow: hidden; - text-overflow: ellipsis; - white-space: nowrap; - text-decoration: none; - } - - .source-link:hover { - color: var(--sl-color-white); - text-decoration: underline; - } - .defaults-summary { + min-width: 0; font-size: var(--sl-text-xs); } .feed-meta { display: flex; flex-wrap: wrap; + justify-content: flex-end; gap: 0.5rem; align-items: center; + align-self: end; font-size: var(--sl-text-xs); } @@ -515,8 +495,8 @@ const staticFeedUrls = configs.map((config) => ({ } .feed-actions { - display: flex; - align-items: center; + justify-self: end; + align-self: start; } .action, @@ -628,18 +608,15 @@ const staticFeedUrls = configs.map((config) => ({ .form-actions { display: flex; + flex-wrap: wrap; + gap: var(--fd-gap); justify-content: flex-end; + align-items: center; } @media (min-width: 50rem) { - .feed-table-head { - display: grid; - } - .feed-row { - grid-template-columns: minmax(0, 1fr) auto; - align-items: center; - gap: 1rem; + gap: 0; } .parameter-form { @@ -657,8 +634,18 @@ const staticFeedUrls = configs.map((config) => ({ grid-template-columns: 1fr; } - .feed-actions { - justify-content: flex-end; + .feed-mainline, + .feed-subline { + gap: 0.375rem; + } + + .feed-subline > :first-child:empty, + .feed-subline > [aria-hidden="true"] { + display: none; + } + + .feed-meta { + justify-content: flex-start; } } From 693436dc526b9434c08394c148799b8860b6b9b9 Mon Sep 17 00:00:00 2001 From: Gil Desmarais Date: Sat, 21 Mar 2026 13:18:16 +0100 Subject: [PATCH 11/11] Delete 2026-03-21-feed-directory-ux-plan.md --- docs/2026-03-21-feed-directory-ux-plan.md | 308 ---------------------- 1 file changed, 308 deletions(-) delete mode 100644 docs/2026-03-21-feed-directory-ux-plan.md diff --git a/docs/2026-03-21-feed-directory-ux-plan.md b/docs/2026-03-21-feed-directory-ux-plan.md deleted file mode 100644 index 4163c1b8..00000000 --- a/docs/2026-03-21-feed-directory-ux-plan.md +++ /dev/null @@ -1,308 +0,0 @@ -# Feed Directory UX Plan - -Date: 2026-03-21 - -## Goal - -Improve the `/feed-directory/` page so the first screen communicates immediate value and shows usable feed results instead of leading with explanatory copy and form-heavy UI. - -This plan is based on the live page at `http://localhost:4321/feed-directory/` inspected with `chrome-mcp`. - -## Current Problems - -### Above the fold - -- Desktop: the first feed is only barely visible at the bottom of the viewport. -- Mobile: no feed is visible above the fold. -- The viewport is dominated by: - - page title - - explanatory intro copy - - a full "Instance URL" documentation section - - two stacked controls before the list becomes useful - -### Feed row design - -- Each row emphasizes internal identifiers (`domain / name`) but hides the more informative `channel.url`. -- The rows feel like utility rows rather than browseable feed entries. -- The `Configure` action is presented as a control, not as context about how the feed is already ready to use. -- Mobile rows are sparse and oversized, with too much empty space around the action buttons. - -## Constraints - -- Do not introduce categories, tags, keywords, or other invented metadata. -- Do not frame any feed as "not ready" because default parameters already make all feeds usable. -- Work only with the fields that exist today: - - `domain` - - `name` - - `channel.url` - - `url_parameters` - - `default_parameters` - - `valid_channel_url` - -## Strategy - -### Above-the-fold strategy - -The first screen should answer three questions immediately: - -1. What is this? -2. Can I use it right now? -3. What should I do first? - -That means the page should open with: - -- little to no explanatory copy above the main interaction -- search as the primary action -- instance URL as contextual secondary state -- actual feed entries visible immediately below - -The long explanation of instance URLs should move below the main browsing experience or into a collapsed/help treatment. - -### Instance URL treatment - -The instance URL should stop behaving like a peer form field to search. - -Recommended treatment: - -- show it as contextual state below search, for example: - - `Using instance: 1.h2r.workers.dev (Change)` -- only reveal the editable input when the user chooses to change it -- reveal the editable input inline rather than using a modal or popover -- visually style it as a small pill, status row, or compact inline setting - -Why: - -- most users will keep the default instance -- two prominent text inputs create unnecessary mental load -- search should own the page, while instance selection should feel like supporting context -- edited instance URLs should be validated inline and gently so a broken value does not silently degrade the experience - -Recommended validation behavior: - -- validate only when the user interacts with the field or attempts to apply the change -- use short inline messaging such as `Enter a valid URL` -- do not block the rest of the page with heavy alerts or modal error states - -### Search treatment - -Search should read like the main job of the page. - -Recommended treatment: - -- larger input -- stronger visual prominence than any other control -- search icon -- generous border radius and spacing so it feels like a global finder, not a small filter field -- placeholder examples grounded in real searches, for example: - - `Search sites or feeds like fia, apple, news, or blog` - -### Feed row strategy - -Each row should explain the feed before asking the user to act. - -Recommended content hierarchy: - -1. `domain / name` -2. source URL from `channel.url` -3. ready-to-use status with defaults inline when present, for example `Ready with defaults: section=news` -4. actions with clear hierarchy: - - `RSS` primary - - `Copy` adjacent to `RSS` - - `Customize` secondary when defaults can be changed - - `Edit` tertiary - -This keeps all feeds framed as ready to use while still exposing how defaults work. - -Recommended action labeling: - -- use `Customize` for feeds whose defaults can be changed -- avoid more technical labels such as `Options` or `Change defaults` -- keep the wording plain and short - -Recommended action styling: - -- `RSS` uses the strongest visual treatment -- `Copy` is visible text on desktop and may collapse to a compact icon treatment on mobile if needed -- `Edit` should be de-emphasized to a ghost or low-contrast tertiary action on desktop and mobile - -### High-value UI refinements - -These additions stay within scope and improve scanability, consistency, and perceived quality: - -- make each feed row feel like a single coherent surface instead of a label with detached controls -- clamp long source URLs to a clean single line where possible instead of allowing messy wrapping -- normalize row height as much as practical so scanning remains fast -- add subtle row hover and focus treatment, not just button hover states -- keep the primary action cluster visually tight so `RSS` and `Copy` read as the main pair -- keep the action anchor consistent across rows so the user learns one reliable target area -- improve typography hierarchy inside `domain / name`, with the feed name carrying slightly more emphasis than the domain - -## Proposed Information Architecture - -### New top section - -- `Feed Directory` -- search-first layout with little to no intro copy above the interaction -- search input -- compact instance URL summary with change affordance -- first feed rows immediately visible - -### Lower-priority content moved down - -- long instance URL explanation -- hosting guidance -- community wiki link -- contribution section - -## Wireframes - -### Desktop - -```text -+---------------------------------------------------------------+ -| Feed Directory | -| [ Search sites, domains, or feed names.................... ] | -| | -| Using instance: 1.h2r.workers.dev [Change] | -+---------------------------------------------------------------+ - -+---------------------------------------------------------------+ -| adfc.de / pressemitteilungen [RSS] [Copy] | -| https://www.adfc.de/presse/pressemitteilungen/ (Edit) | -+---------------------------------------------------------------+ -| apnews.com / hub [RSS] [Copy] | -| https://apnews.com/%
s [Customize] | -| Ready with defaults: section=news (Edit) | -+---------------------------------------------------------------+ -| avherald.com / index [RSS] [Copy] | -| https://avherald.com/ (Edit) | -+---------------------------------------------------------------+ -``` - -### Mobile - -```text -+--------------------------------------+ -| Feed Directory | -| [ Search........................... ]| -| Using: 1.h2r.workers.dev [Change] | -+--------------------------------------+ - -+--------------------------------------+ -| adfc.de / pressemitteilungen | -| https://www.adfc.de/presse/... | -| [RSS] [Copy] (Edit) | -+--------------------------------------+ - -+--------------------------------------+ -| apnews.com / hub | -| https://apnews.com/%
s | -| Ready with defaults: section=news | -| [RSS] [Copy] [Customize] (Edit) | -+--------------------------------------+ -``` - -## Implementation Plan - -### 1. Restructure page content - -Update `src/content/docs/feed-directory/index.mdx`: - -- replace the current intro paragraph plus full "Instance URL" section with a near action-first layout -- move longer explanatory content below the directory component -- keep contribution content lower on the page -- update `Contribute on GitHub` to link directly to: - - `https://github.com/html2rss/html2rss-configs/tree/master/lib/html2rss/configs` - -### 2. Redesign top controls - -Update `src/components/FeedDirectory.astro`: - -- make search the primary visible control -- demote instance URL into a compact secondary control -- hide the editable instance input behind a change affordance or disclosure -- reduce vertical space before the list begins -- style search as the visual hero of the page -- keep the top section simple and avoid introducing extra explanatory UI - -### 3. Redesign feed rows - -Update `src/components/FeedDirectory.astro`: - -- show `channel.url` as visible secondary context -- show inline ready-to-use default context for configurable feeds -- keep `RSS` as the strongest action -- add a `Copy` action beside `RSS` -- reduce the prominence of `Edit`, ideally to a ghost or tertiary action -- rename or reposition `Configure` so it reads as optional customization, not a gate -- use `Customize` as the action label for changing defaults -- make the full row read as one card-like unit -- tighten the visual grouping of primary actions -- improve title typography so rows are easier to scan -- clamp or truncate long source URLs cleanly - -### 4. Refine mobile layout - -- reduce excessive vertical gaps -- reduce top chrome pressure as much as possible because the Starlight shell already consumes significant vertical space -- keep row content denser without hurting tap targets -- ensure at least one full feed row is visible above the fold on common mobile viewports -- avoid rows becoming disproportionately tall because of long text or wrapping actions -- prefer horizontal action grouping that stays on one line -- keep at least one full card plus part of the next card visible above the fold where possible so continuation is obvious - -### 5. Improve empty states - -Update `src/components/FeedDirectory.astro` and `src/components/feed-directory.js`: - -- when search returns no matches, show a visible empty state instead of a blank list -- include the active query in the message -- point users toward the community wiki or contributing a new configuration -- include a clear next step, not just a passive β€œno results” message - -### 6. Add interaction polish - -Update `src/components/FeedDirectory.astro` and `src/components/feed-directory.js`: - -- show immediate feedback on copy actions, for example `Copied` for 1-2 seconds -- ensure keyboard focus styles are crisp and obvious, especially around search and row actions -- keep any transitions very light so the page feels fast and utilitarian rather than decorative -- validate edited instance URLs inline with gentle feedback rather than failing silently -- prefer inline disclosure patterns over popovers or modal flows - -### 5. Preserve existing behavior - -Update `src/components/feed-directory.js` only as needed: - -- keep search behavior -- keep instance URL updates -- keep parameter editing behavior -- make sure default-parameter feeds still generate working RSS links immediately - -## Acceptance Criteria - -- Desktop shows search and at least the first feed row clearly above the fold. -- Mobile shows search and at least one complete feed row above the fold. -- The page communicates value before documentation. -- Feed rows expose useful context, not just identifiers and buttons. -- Feeds with defaults are presented as immediately usable. -- Search is visually dominant over instance configuration. -- Empty searches produce a helpful message instead of an empty list. -- Feed rows feel visually cohesive and easy to scan. -- Long URLs do not degrade the row layout. -- Edited instance URLs provide clear inline feedback when invalid. -- Mobile actions remain on a single line in normal cases. -- Empty states provide a next step. -- Copy actions acknowledge success immediately. -- Keyboard focus is visually obvious. -- Motion remains subtle and fast. -- `Contribute on GitHub` points directly to the configs directory in the repository. -- No invented metadata is introduced. - -## Notes From Live Review - -Observed on the live local page with `chrome-mcp`: - -- Desktop viewport was consumed by heading, intro copy, the full "Instance URL" explanation, and form controls before feed content became visible. -- Mobile viewport showed no actual feed row above the fold. -- Feed rows lower on the page showed that the current design underuses the available data, especially `channel.url`.