Git Product home page Git Product logo

obsidian-repos-downloader's Introduction

obsidian-repos-downloader

Contents

on-push-do-doco

What?

Clone every approved Obsidian.md community Plugin and Theme - to read and search the source code and learn from the community.

This is a Python3 script to download a local copy of all the published community Obsidian plugins and themes, to be used as a large body of example code.

It inspects these files, and then downloads (clones) all the repos listed in them:

Why?

I cannot put it better than the author of the similar project luckman212/obsidian-plugin-downloader:

As an absolute beginner to TypeScript, and a lover of Obsidian I often want to take a look at how someone has achieved a certain feature, called on an API, etc. A quick way to do that is by searching through the existing codebase of the ever growing library of plugins out there.

Setup

Requirements

  • Python 3.6 or above

Download

  1. Download the Latest Release.
    • Choose one of:
      • "Source code (zip)"
      • "Source code (tar.gz)"
    • If you can't see them, click to expand the "Assets"
  2. Expand the downloaded Source Code file
    • This will give you a folder name such as "obsidian-repos-downloader-0.1.0"

Run

Getting Started

The script to run is obsidian-repos-downloader.py

Depending on your platform, here are some example ways you might need to run it:

obsidian-repos-downloader.py
./obsidian-repos-downloader.py
python3 obsidian-repos-downloader.py

Usage - all the arguments

Running obsidian-repos-downloader.py --help gives this output:

usage: obsidian-repos-downloader.py [-h] [-o OUTPUT_DIRECTORY] [-l LIMIT] [-n]
                                    [-t [{plugins,themes,all}]]
                                    [--group-by-user] [--no-group-by-user]

Clone repos included in the obsidian-releases repo, to provide a body of
example plugins and CSS themes.

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT_DIRECTORY, --output_directory OUTPUT_DIRECTORY
                        The directory where repos will be downloaded. Must
                        already exist. (default: . which means "current
                        working directory")
  -l LIMIT, --limit LIMIT
                        Limit the number of plugin and theme repos that will
                        be downloaded. This is useful when testing the script.
                        0 (zero) means "no limit". Note: the count currently
                        includes any repos already downloaded.(default: 0)
  -n, --dry-run         Print out the commands to be executed, but do no run
                        them. This is useful for testing. Note: it does not
                        print the directory-creation commands, just the git
                        ones
  -t [{plugins,themes,all}], --type [{plugins,themes,all}]
                        The type of repositories to download: plugins, themes
                        or both. (default: all)
  --group-by-user       Put each repository in a sub-folder named for the
                        GitHub user. For example, the plugin
                        "https://github.com/phibr0/obsidian-tabout" would be
                        placed in "plugins/phibr0/obsidian-tabout"
  --no-group-by-user    Put each repository in the same folder, prefixed by
                        the user name. This is the default behaviour. For
                        example, the plugin
                        "https://github.com/phibr0/obsidian-tabout" would be
                        placed in "plugins/phibr0-obsidian-tabout"

Output Directories

The script always creates a plugins/ and themes/ directories for its output.

There are the command-line arguments to determine the structure inside those directories.

Flatter Structure

By default, or when the argument --no-group-by-user is supplied, all the downloaded repos are placed side-by-side. They are prefixed with the username of the developer who wrote them.

For example, running this command (limiting the output to only 4 repositories, for brevity)....

obsidian-repos-downloader.py  --limit 4

... gives this directory structure:

plugins
├── agathauy-wikilinks-to-mdlinks-obsidian
├── aidenlx-alx-folder-note
├── aidenlx-better-fn
└── aidenlx-cm-chs-patch
themes
├── ArtexJay-Obsidian-CyberGlow
├── auroral-ui-aurora-obsidian-md
├── bcdavasconcelos-Obsidian-Ayu
└── bcdavasconcelos-Obsidian-Ayu_Mirage

8 directories

Grouped by User name

When the argument --group-by-user is supplied, all the downloaded repos are placed in sub-directories named with the username of the developer who wrote them.

For example, running this command (limiting the output to only 4 repositories, for brevity)....

obsidian-repos-downloader.py  --limit 4 --group-by-user

... gives this directory structure:

plugins
├── agathauy
│   └── wikilinks-to-mdlinks-obsidian
└── aidenlx
    ├── alx-folder-note
    ├── better-fn
    └── cm-chs-patch
themes
├── ArtexJay
│   └── Obsidian-CyberGlow
├── auroral-ui
│   └── aurora-obsidian-md
└── bcdavasconcelos
    ├── Obsidian-Ayu
    └── Obsidian-Ayu_Mirage

13 directories

Likely Questions

How do I update repos I have already downloaded?

This is now done automatically, via git pull, for repos that have already been cloned.

What order are plugins and themes downloaded in?

They are downloaded in case-insensitive alphabetical order of the repository's GitHub URL, so effectively in order of user name and then repo name.

What if there is an error?

Sometimes it is not possible to update a repo, for example of there are edited files on the local machine, or the name of the remote branch has changed (such as from 'master' to 'main').

The script accumulates a list of errors, and prints them on completion.

The easiest way to deal with such errors is to delete the downloaded repo, and run the script again.

Example error output:

The following errors occurred:
updating Slowbad/obsidian-solarized
command:    git pull --quiet
in:         /Users/clare/obsidian-repos-downloader/themes/Slowbad-obsidian-solarized
exit code:  1
stdout:     
stderr:     Your configuration specifies to merge with the ref 'refs/heads/master'
from the remote, but no such ref was fetched.

-------------------------------------------------------------------------------

Alternatives

There is a growing number of alternative mechanisms for downloading Obsidian repos:

obsidian-repos-downloader's People

Contributors

claremacrae avatar actions-user avatar

Stargazers

TheMattaBase avatar MattV avatar a13ph avatar Joel Van Eenwyk avatar  avatar  avatar Mariusz Korzekwa avatar Judson Ennis avatar  avatar Blazej Gruszka avatar Tong avatar Moritz Jung avatar vancaem avatar ohwthrandi avatar hdtracy avatar Keith Adams avatar  avatar david.dev.br avatar JohnnyLearns avatar  avatar  avatar Chris Grieser avatar Jan Szymański avatar

Watchers

James Cloos avatar  avatar  avatar

Forkers

joelvaneenwyk

obsidian-repos-downloader's Issues

FR: restrict to `.ts` / `.css` files

I was pleased to see that this tool can also be used with themes!

So, I noticed that the download takes quite a bit of time, especially since it downloads the full repos, which also include the high-res promo screenshots from the themes, or demo-videos from the docs from plugins.

Since the purpose of this tool is primarily to inspect code and not screenshots, how about only downloading .ts and .css (or .scss)? this would immensely speed up the download, and also save bandwidth & file space.

There could be an option to still download the full repos if so wished by the user

Trap any errors and continue, displaying them at the end

Even on a fast, seemingly stable internet connection, I get random errors when updating...

It's annoying - it would be more convenient to continue past the errors, and print a summary at the end.

Example output:

updating Reocin/obsidian-markdown-formatting-assistant-plugin
error: cannot pull with rebase: You have unstaged changes.
error: please commit or stash them.
Traceback (most recent call last):
  File "./obsidian-repos-downloader.py", line 174, in <module>
    download_repos()
  File "./obsidian-repos-downloader.py", line 170, in download_repos
    downloader.download()
  File "./obsidian-repos-downloader.py", line 100, in download
    self.process_released_plugins()
  File "./obsidian-repos-downloader.py", line 104, in process_released_plugins
    self.process_released_repos("plugins", PLUGINS_JSON_FILE)
  File "./obsidian-repos-downloader.py", line 117, in process_released_repos
    self.clone_repos(sorted_list)
  File "./obsidian-repos-downloader.py", line 123, in clone_repos
    self.clone_repo(plugin)
  File "./obsidian-repos-downloader.py", line 142, in clone_repo
    self.run_or_log(f"updating", command, repo)
  File "./obsidian-repos-downloader.py", line 149, in run_or_log
    subprocess.run(command, shell=True, check=True)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'git pull --quiet' returned non-zero exit status 128.

Ah - just read the error message, and it's not a network error!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.