html2rss / html2rss-configs Goto Github PK

View Code? Open in Web Editor NEW

20.0 2.0 7.0 346 KB

📇 A growing collection of html2rss feed configs. Generate configs with the handy generator.

Home Page: https://html2rss.github.io/configs/

License: MIT License

Ruby 97.39% Shell 0.58% Makefile 2.02%

html2rss configuration html2rss-web presets scraping html2rss-configs feed feed-configs dynamic-parameters

html2rss-configs's Introduction

html2rss-configs

✌️This repository contains html2rss feed configs for many websites. 👉Find all feed configs in the configs/ directory. ☝️A handy usage method is via html2rss-web. 💪 Contributions are more than welcome! Fork this repository, add your feed config and create a pull request.

Adding a new feed config

There's a generator for that! 🆒

Fork this repo and run bundle install (you need to have Ruby >= 3.1 installed).
Start the generator by typing: bin/generator
Build your feed config and answer 'y' in the last step to create the files.
Optionally, edit the created files. Read html2rss's README what is possible or browse existing configs for inspiration.
To test, run: bundle exec html2rss feed lib/html2rss/configs/domainname.tld/whatever

Using dynamic parameters in `channel` attributes

When you're using dynamic parameters, you have to provide the parameters to the spec, too:

include_examples 'config.yml', 'domainname.tld/whatever.yml', id: 42

CLI usage:

bundle exec html2rss feed lib/html2rss/configs/domainname.tld/whatever id=42

Programmatic usage

Add to your Gemfile:

gem 'html2rss-configs', git: 'https://github.com/html2rss/html2rss-configs.git'

Use it in your code:

require 'html2rss/configs'

config = Html2rss::Configs.find_by_name('domainname.tld/whatever')

This will return the feed config.

Usage with `html2rss-web`

If you're running html2rss-web, you have nothing more to do! 🎉

Building on the CI

Modifying existing or adding new feed configs will trigger the CI to fetch the feed and check for the presence of feed items.

See .travis.yml -> script which commands execute during build.

html2rss-configs's People

Contributors

Stargazers

Watchers

Forkers

fl4min danganhhop polyeto nawordar vikingdadmedic qidouhai

html2rss-configs's Issues

Extract generator to its own, installable, gem

use as dev dependency in h2r-configs

improve the config generator

The feed config generator, in its current state, is a bit clunky to use. The rails world is full of generators which scaffold some files. So, I imagine, having rake config https://example.com/path could automatically:

create the feed config
create the feed config's spec
could prefill some values in the feed config (channel title (from page's <title>)), language, etc (open graph and meta tags)

In another iteration it could also suggest the selectors, but that's a topic for another day.

ArgumentError when trying to generate a new config

I have tried to generate a new configuration, but when I ran bin/generator it immediately crashed with the following error:

nawordar ~/html2rss-configs (master)
❯ bin/generator
/home/nawordar/.gem/ruby/3.0.0/gems/bundler-1.17.3/lib/bundler/shared_helpers.rb:29: warning: Pathname#untaint is deprecated and will be removed in Ruby 3.2.
/home/nawordar/.gem/ruby/3.0.0/gems/bundler-1.17.3/lib/bundler/shared_helpers.rb:118: warning: Pathname#untaint is deprecated and will be removed in Ruby 3.2.
/home/nawordar/.gem/ruby/3.0.0/gems/bundler-1.17.3/lib/bundler/shared_helpers.rb:118: warning: Pathname#untaint is deprecated and will be removed in Ruby 3.2.
/home/nawordar/.gem/ruby/3.0.0/gems/bundler-1.17.3/lib/bundler/shared_helpers.rb:35: warning: Pathname#untaint is deprecated and will be removed in Ruby 3.2.
/home/nawordar/.gem/ruby/3.0.0/gems/bundler-1.17.3/lib/bundler/shared_helpers.rb:35: warning: Pathname#untaint is deprecated and will be removed in Ruby 3.2.
/home/nawordar/.gem/ruby/3.0.0/gems/bundler-1.17.3/lib/bundler/shared_helpers.rb:44: warning: Pathname#untaint is deprecated and will be removed in Ruby 3.2.
/home/nawordar/.gem/ruby/3.0.0/gems/bundler-1.17.3/lib/bundler/shared_helpers.rb:118: warning: Pathname#untaint is deprecated and will be removed in Ruby 3.2.
html2rss config generator

This wizard will help you create a new feed config.

You can quit at any time by pressing Ctrl+C.

/home/nawordar/html2rss-configs/.bundle/ruby/3.0.0/gems/tty-prompt-0.23.0/lib/tty/prompt.rb:201:in `ask': wrong number of arguments (given 2, expected 0..1) (ArgumentError)
        from /home/nawordar/html2rss-configs/lib/html2rss/configs/generator/question.rb:21:in `ask'
        from /home/nawordar/html2rss-configs/lib/html2rss/configs/generator/collector.rb:35:in `each'
        from /home/nawordar/html2rss-configs/lib/html2rss/configs/generator/collector.rb:35:in `collect'
        from /home/nawordar/html2rss-configs/lib/html2rss/configs/generator.rb:21:in `start'
        from bin/generator:7:in `<main>'

I am new to Ruby, so I'm not sure if I configured everything correctly. I ran the following commands before starting the generator:

gem install bundler:1.17.3
bundle install --path .bundle

EDIT Never mind, I used too new Ruby version: 3.0.0. Using asdf-vm to use 2.7.4 fixed the problem

migrate from travis to github actions

TravisCI.org is still connected, but somehow not sending information to PRs. As I had some more struggles with travis recently, I'd like to move the CI to Github Actions.

Dependency Dashboard

This issue provides visibility into Renovate updates and their statuses. Learn more

This repository currently has no open or pending branches.

Check this box to trigger a request for Renovate to run again on this repository