radian-software / straight.el Goto Github PK

We currently prune the build cache on a successful init. We should also prune the build directory. This should be easy to implement, and harmless.

We should automatically remove no-longer-present packages from versions.el each time it is written using straight-save-versions.

M-x straight-freeze-versions is supposed to install all packages, even ones not needed on the current system. Therefore, there needs to be a way to register a package with straight.el without actually installing it.

The current algorithm is O(n) in the number of packages at Emacs init. We could reduce this to O(n) in the number of packages loaded at Emacs init, if the user enables an option to only check packages for rebuilds when they are actually loaded. This option should be disabled by default since it requires putting advices on require and load.

One potential complication is that it's not clear how we could figure out what features are supposed to be require-able, if the packages are not built yet. Perhaps this indicates that #41 is a better way to improve performance.

Fix it:

(defun straight--update-package (recipe)
  ;; FIXME
  (error "Don't know how to update packages yet"))

We need some proper documentation in the README.

Currently that has no effect.

Why?

OBSOLETE: 1.0 release checklist

This is a tracking issue for issues that need to be closed for the 1.0 release of straight.el:

• Improve user experience around package building:
  • Fix slowness while reloading init-file (#35).
  • Fix duplicate warnings on changing recipe (#34).
  • Investigate whether packages are built more than once during
    init (#27).
  • Add straight-rebuild and straight-rebuild-all commands (#10).
  • Dependencies are screwed up when installing after init.
  • Install even built-in packages (#38).
  • Record individual packages in bulk find(1) operation (#42).
  • Don't load package.el at runtime (#43).
  • Support registering a package without installing it (#44).
  • Make sure straight-freeze-versions installs registration-only packages before freezing.
  • Figure out why ESS wants to make autoloads (#61).
  • Automatically prune build directory (#50).
  • Build texinfo manuals (#71).
  • Support other build commands (#72).
• Make version-control support more customizable.
  • Support upstream/fork workflow (#29).
  • Remove pbl (#25).
    • Fix superfluous symlinking (#63).
  • Support pluggable version-control backends.
  • Install versions specified in lockfile by default (#23).
    • Clone correct version of straight.el on initial setup (#56).
    • Allow to specify a custom recipe for straight.el (#57).
  • Support deferred installation in straight-freeze-versions (#24).
  • Add straight-override-package (#39).
  • Fix bugs introduced by new version-control support.
    • Fix issues with default-directory in popup system (#53).
    • Improve UX around pushing (#54).
    • Show unmerged commits when considering pull or push (#55).
    • Fetch if needed in straight-thaw-versions (#58).
    • Eliminate nil branch name error (#59).
    • Fix byte-compilation staleness (#60).
    • Remove option to set remote URLs (#64).
    • Set pushRemote and/or pushDefault when validating remotes (#65).
    • Move away from decapitation (#66).
• Support pluggable recipe repositories (#30).
  • Support org-elpa (#36).
  • Support elpa externals-list (#18).
  • Figure out why profile-dotemacs doesn't show up in Emacsattic (#52).
  • Save versions of recipe repositories in lockfile (#26).
  • Use heuristics from borg.el to compute EmacsMirror recipes (#40).
  • Prefix argument should make straight-get-recipe prompt for recipe source (#62).
• Improve performance (#9):
  • Add option to only rebuild packages when changed in Emacs (#41).
• Improve use-package integration:
  • Make use-package-install-deferred-package work (upstream, see jwiegley/use-package#479).
  • Integrate registration-only package usage with use-package (#45).
• Miscellaneous tasks:
  • Don't expose the internal arguments of straight-use-package (#46).
  • Add Travis to check byte-compilation (#19).
    • Run checkdoc on Travis (#75).
    • Check for long lines on Travis (#77).
  • Allow overriding a package's commit hash in your own lockfile (#67).
  • Neuter package.el (#73).
• Add comprehensive documentation (#28, #51).
  • Add example configurations (#74).

straight.el needs to manage itself as a package. Otherwise it remains unversioned in versions.el, and also pbl does not have its autoloads activated.

Although the profile system allows for adding your own packages without putting them in other lockfiles, there is no way of overriding the revision of a package from another profile. This can be corrected with a new function straight-override-package, which will also allow for overriding the recipe ahead of installation time.

Jonas uses some heuristics (for example, to calculate the load path) in order to handle the variety of packages on the EmacsMirror. These should be incorporated into straight.el, since otherwise some packages from EmacsMirror will not be usable without providing a custom :files directive.

Version control code should be self-hosted. We can drop support for non-Git VCS systems.

MELPA/GNU ELPA/Emacsmirror is hardcoded in. This is bad, it should be extensible.

Currently, straight--set-head will just fail if you have unstaged changes. It should offer to do a hard reset, clean the working directory, and so on, if necessary.

My startup time was 0.5s before using straight.el; now it's 0.9s.

Currently dependencies are not necessarily explained very well, since only the direct cause is reported (e.g. a dependency could cause MELPA to be cloned, and the message displayed is then unhelpful). I would like the whole chain to be shown.

All packages are built on every init.

Here's another problem I noticed with it. If you delete the build directory, it fails to notice and errors out trying to load the autoload file.

It seems that evaluating a use-package form causes a package rebuild every time.

This would rebuild any packages that needed to be rebuilt. Handy if you're testing something like esup where the new instance of Emacs won't pick up your changes because it's bypassing the normal init process.

Currently if you evaluate a use-package form and then change its recipe, you will just get an error because the recipe has changed. This is arguably correct but definitely annoying. There needs to be a way to handle this without resorting to mutating multiple undocumented internal hash tables.

During init, the same package is built more than once.

We need docstrings.

Cloning Git repos can take a while. It would be nice to offer a config option to clone repositories shallowly by default.

After #44 is implemented, we'll want to automatically use it when the use-package keyword :if is present.

It's probably better to keep track of TODO items in a proper issue tracker rather than having lots of scattered FIXME notices.

After #25, we may want to consider enhancing the Git support. In particular, I'd like explicit support for a fork-based workflow. Something like:

(straight-use-package
  '(use-package
    :repo (github . "raxod502/use-package")
    :upstream (github . "jwiegley/use-package")))

It would certainly be possible to automatically transform MELPA recipes into this style, so this is doable.

ಠ_ಠ

We need package.el for package-built-in-p at runtime, so we know whether or not to signal an error due to a missing package. Unfortunately, this means we now have a runtime dependency on package.el. I think the most amusing way to solve this problem is to persistently cache the complete list of built-in packages, and update the cache whenever we detect that the output of M-x emacs-version has changed.

Over time, you can accrue a large number of unused Git repositories that were cloned at various points but are no longer used. I don't want to prune these (they might contain useful development history), but some people who don't modify their Emacs packages might.

I just realized that versions.el is going to have not only Radian's packages, but also the packages I declare in my local dotfiles. These need to be separate, so straight.el needs to have an idea of multiple package declaration sources, which can be stored in separate versions.el files.

Like melpa-stable.

If the recipe specifies a branch or particular revision, that needs to be checked as well.

Even if there are no unit tests or documentation yet, I would at least like to validate that there are no byte-compiler warnings.

If a package depends on Org, it probably depends on the latest version, not whatever version is shipped with Emacs. Even if package.el insists a package is built-in, we should still try to install it. The arguably most correct solution would be to attempt to install all packages, but then only signal an error on missing dependencies if those dependencies are built-in. I'm tempted to just say that missing dependencies are not an error, given how loose the rest of dependency handling in straight.el is. But this is probably going too far.

Currently, straight--get-head and straight--set-head only work for Git. They should also work for other VCS's.

I think this is a pretty simple problem but I don't currently have time to debug it.

If I evaluate a straight-use-package form after init, then the dependencies don't seem to get properly saved in the build cache, and so they are not activated the next time Emacs is started.

This could also be a problem with another Emacs instance overwriting the build cache. That is a problem that needs to be solved! Perhaps an intelligent merge could be done. Although, I think the build cache should be loaded and saved only for the evaluation, not on Emacs kill.

Currently, if any packages have changed, we discard the results of our bulk find(1) operation (from straight--cached-packages-might-be-modified-p) and check each package individually. This is slower. It would be preferable to stick the results in a hash table so that we can skip checking all cached packages, even if some are modified.

straight.el should save the recipe in build-cache.el and rebuild the package if the recipe has changed by the next check.

I got this:

Warning (straight): Packages "esup" and "esup" have incompatible recipes (:branch cannot be both "radian-1" and nil)
Warning (straight): Package "esup" has two incompatible recipes (:branch cannot be both "radian-1" and nil)

I think the first warning is superfluous.

Currently if there is an error during init, all the packages will be rebuilt next init because the build cache is written to disk (and not all of the packages had been enabled yet).

This would help with #9. Probably we should also handle some simple cases like the build directory or repository being deleted, which users might try if they didn't know about straight-rebuild-package.

I think we can move the current definition of straight-use-package to straight--use-package, and then expose a new version of straight-use-package that omits the arguments currently marked as "internal use only".