cristibalan / braid Goto Github PK

If the version braided into a project is not present in the local cache, braid should synchronize the mirror and try again. Historically it used to always synchronize the mirror so this was never a problem.

Is it possible to track a other branch while tracking master?

IE:
We are following branch REL1_23, and we want to update/merge it to REL1_24?

Is partial checkout a supported use case? My use case needs a single directory from the imported repo and not the whole thing. I cannot find anything in the docs about this (maybe I'm missing this).

Help would be appreciated!

Currently braid adds remotes for every upstream repository in the downstream repository. You end up with remotes looking something like the following

$ git remote -v
master/braid/vendor/docs/way_of_stock	/Users/peter/.braid/cache/https___github.com_stocksoftware_way_of_stock.git (fetch)
master/braid/vendor/docs/way_of_stock	/Users/peter/.braid/cache/https___github.com_stocksoftware_way_of_stock.git (push)
master/braid/vendor/plugins/dbt	/Users/peter/.braid/cache/https___github.com_realityforge_dbt.git (fetch)
master/braid/vendor/plugins/dbt	/Users/peter/.braid/cache/https___github.com_realityforge_dbt.git (push)
master/braid/vendor/plugins/domgen	/Users/peter/.braid/cache/https___github.com_realityforge_domgen.git (fetch)
master/braid/vendor/plugins/domgen	/Users/peter/.braid/cache/https___github.com_realityforge_domgen.git (push)
master/braid/vendor/plugins/rptman	/Users/peter/.braid/cache/https___github.com_realityforge_rptman.git (push)
master/braid/vendor/plugins/rptman	/Users/peter/.braid/cache/https___github.com_realityforge_rptman.git (fetch)
master/braid/vendor/tools/buildr_plus	/Users/peter/.braid/cache/https___github.com_realityforge_buildr_plus.git (push)
master/braid/vendor/tools/buildr_plus	/Users/peter/.braid/cache/https___github.com_realityforge_buildr_plus.git (fetch)
master/braid/vendor/tools/dbt	/Users/peter/.braid/cache/https___github.com_realityforge_dbt.git (push)
master/braid/vendor/tools/dbt	/Users/peter/.braid/cache/https___github.com_realityforge_dbt.git (fetch)
master/braid/vendor/tools/domgen	/Users/peter/.braid/cache/https___github.com_realityforge_domgen.git (push)
master/braid/vendor/tools/domgen	/Users/peter/.braid/cache/https___github.com_realityforge_domgen.git (fetch)
master/braid/vendor/tools/kinjen	/Users/peter/.braid/cache/https___github.com_realityforge_kinjen.git (push)
master/braid/vendor/tools/kinjen	/Users/peter/.braid/cache/https___github.com_realityforge_kinjen.git (fetch)
master/braid/vendor/tools/redfish	/Users/peter/.braid/cache/https___github.com_realityforge_redfish.git (fetch)
master/braid/vendor/tools/redfish	/Users/peter/.braid/cache/https___github.com_realityforge_redfish.git (push)
master/braid/vendor/tools/rptman	/Users/peter/.braid/cache/https___github.com_realityforge_rptman.git (fetch)
master/braid/vendor/tools/rptman	/Users/peter/.braid/cache/https___github.com_realityforge_rptman.git (push)
...

Regular users who interact with the downstream repository find this difficult to work with, particularly when the number of braided directories gets high and/or they are already working with different remotes themselves. It also gets very complex if the braided repositories have tags as they pollute the downstream repository and users who do things such as git push --tags end up with behaviour they really do not expect.

An alternative would be to not add the remotes explicitly and instead interact with the cache via local urls. Another alternative would be to add and then remove remotes as part of the braid process. Either option would improve the user experience at the expense of a more complex braid codebase.

Should we be remembering to update http://cristibalan.github.io/braid every time we update the readme, or shall we just redirect it to the repository so people can see the readme (and add any desired download links there)?

First of all, good that there is some activity on this project, I thought it has come to rest some time ago...

Having last release in 2011 makes really suboptimal impression... Maybe we could release something working?
https://rubygems.org/gems/braid

VERSIONS:
0.7.1 - August 25, 2011 (27 KB)
0.7.0 - March 15, 2011 (26.5 KB)
0.6.2 - July 18, 2010 (26 KB)
0.6.1 - October 28, 2008 (26 KB)
0.6 - October 28, 2008 (25.5 KB)

even the current version would probably do...

Cheers

The latest version of braid (1.0.4) with the changes from @mattmccutchen has started to cause conflicts with files outside the braided directory. The scenario seems to come about when you have multiple braids into a single repository. So it is not uncommon for our projects to set up a braids for a set of projects such as;

• vendor/tools/tool1
• vendor/tools/tool2
• vendor/tools/tool3

And frequently when you delete several files from tool1 and then braid update vendor/tools/tool1 you will end up with conflicts in directories vendor/tools/tool2 or vendor/tools/tool3 as braid tries to delete them. (Presumably the hash collides?)

This only happens in braid (1.0.4) - @mattmccutchen do you have any ideas about what would be causing this and/or how to fix it?

If you try to add a braid and there is an error during git fetch, braid will not tell you what it is, even if you turn on verbose mode. This makes determining why braid isn't working harder than it needs to be.

Example:

$ braid add git://git.apache.org/buildr.git vendor/buildr -v
Braid: Executing `git --version`
Braid: Executing `git status`
Braid: Executing `git rev-parse HEAD`
Braid: Adding git mirror of 'git://git.apache.org/buildr.git'.
Braid: Executing `git --version`
Braid: Executing `git config remote.braid/vendor/buildr.url`
Braid: Executing `git config svn-remote.braid/vendor/buildr.url`
Braid: Setup: Creating remote for 'vendor/buildr'.
Braid: Executing `git remote add -t master -m master braid/vendor/buildr /Users/rsutphin/.braid/cache/git___git.apache.org_buildr.git`
Braid: Executing `git config remote.braid/vendor/buildr.url`
Braid: Executing `git fetch  2>&1 >/dev/null`
Braid: Executing `git fetch -n braid/vendor/buildr 2>&1 >/dev/null`
Braid: Resetting to 'b3eca84'.
Braid: Executing `git reset --hard b3eca84da9af3175978d05d59461ca3cbecce6c9`
Braid: Shell error: could not fetch

By modifying braid to remove the redirects to /dev/null I was able to track the problem down (apache git mirrors don't have a "master" branch, so I needed to specify a branch explicitly), but it took a while to get there. If the full fetch output were provided in verbose mode (or, better, whenever there was an error), the problem would have been clear right away.

Hi,
I've installed braid 0.6 on my laptop with Snow Leopard and on attempt to exacure app i see following

C4s-MacBook-Pro:Temp alec$ braid -v
/opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in gem_original_require': no such file to load -- /opt/local/lib/ruby/gems/1.8/gems/braid-0.6/lib/core_ext (LoadError) from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:inrequire'
from /opt/local/lib/ruby/gems/1.8/gems/braid-0.6/lib/braid.rb:23
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in gem_original_require' from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:inrequire'
from /opt/local/lib/ruby/gems/1.8/gems/braid-0.6/bin/braid:4
from /opt/local/bin/braid:19:in `load'
from /opt/local/bin/braid:19

What is this?

Is there any way to do a post update hook with braid? Say I wanted to have it move some files to another directory after the update is run, can i do that?

If braid update fails to update a repository it should halt progress. Otherwise the .braids.json file can get out of date.

Sometimes a user wants to braid update an existing upstream or braid add a new upstream and if there is no local changes in the local path this should be possible but it is not currently. It may require a little bit of tricky coding and there is the risk that in the case that braid update causes a conflict you may lose your local index but this may be acceptable.

@cristibalan Can you enable "GitHub Pages" for the braid project. It is done via the settings page associated with the project. Then just auto-generate a site based on the README.md and select a style to go with it. Can you also add the target url as the website of the project.

I am unable to do this and I believe only the owner of the project can do this.

Thanks!

When I started using braid, I chose the vendor/ directory. But then I decided to move my folders to public/assets/javacripts/vendor instead. Now when I try to remove the original path from braid list, it keeps giving me this error:

us150803:magellan patchuser$ braid remove vendor/assets/chosen/
Braid: Removing mirror from 'vendor/assets/chosen'.
Braid: Resetting to 'd5d516e'.
Braid: Shell error: fatal: pathspec 'vendor/assets/chosen' did not match any files

Any ideas how to resolve this?

I was looking for a subproject management tool that puts the subproject content in the main tree (like git subtree) and keeps an explicit record of the upstream commit so it doesn't mess up on revert, merge, and rebase (unlike git subtree). (Full thread.) Unfortunately I didn't find out about Braid until after I wrote my own bare-bones tool, but now I'm looking forward to switching to Braid if I can fix the few important deficiencies compared to my tool.

So, the first thing I saw when reviewing the implementation of braid update: it generates temporary commits by actually running a git commit command in the user's worktree on the user's branch, so I assume that if it dies in the middle for any reason, it can leave the worktree and branch in an inconsistent state. Instead, it should use a temporary index file.

If I don't hear any objections, I plan to try to implement this and send a pull request in the next few days.

Sometimes end users like to inspect and interact with the .braids file directly and would like their editor to detect it as json. A simple way to do this would be to rename the file from .braids to .braids.json. This could be done in braid update command in a similar fashion to the way we moved from yaml to json.

There is a large quantity of code in bin/braid. My view is that large bins in gems are an anti-pattern.

Most of that code should be moved into lib. This is usually accomplished by making a class like Braid::Cli and sticking it into lib/braid/cli.rb. This file would typically not need to be required in lib/braid.rb. Instead, Both braid and braid/cli would be required in bin/braid.

Once all the code is moved into a class, the content of bin/braid would be just a set of requires and then a single line like Braid::Cli.run(ARGV).

Hi

braid diff should:

• Accept options for git diff that don't interfere with what Braid is doing.
• Run git diff directly on the terminal so that the same pager and color settings apply as a normal git diff.

The Braid::Operations::Proxy::exec method tries to read the subprocess's stdout to EOF before it reads the stderr, so it deadlocks if the subprocess tries to write more to stderr than the OS's buffer size before closing stdout.

The case I ran into was adding a mirror of an upstream repository that has many refs, so the listing of fetched refs printed by git fetch to stderr gets very long. Specifically, this reproduces the problem for me:

braid add -r ef3ad82a04edc164555c619575ffa0716e6258be -p packages/minimongo https://github.com/meteor/meteor packages/minimongo

An easy change that avoids the problem in this case (and which we may want anyway): add the --quiet option to git fetch.

However, we should probably implement a general-purpose function like Braid::Operations::Proxy::exec correctly. A quick web search found Open3.capture3, which is exactly what we want if it's available in all the Ruby versions that Braid intends to support. @realityforge, what versions are these? We should document the Ruby version assumption, ideally in the gemspec, although I don't know if what we want to say about regular Ruby vs. JRuby can be expressed in the gemspec.

Sometimes a user wants to update to use a particular branch of the mirror. Currently this requires editing of the .braids.json file. It should be exposed via the braid update command or maybe braid switch or similar

I'd like to be able to add, update, or remove mirrors in the same commit as other related changes. I propose that the add, update, and remove commands could take a --no-commit option to stage the changes without automatically committing. This option should also allow the working tree to have local changes as long as they won't interfere with the command.

I don't think this is a big enough deal for me to put in the work to implement it right now (I can just make multiple commits and then squash them), but I wanted to go ahead and log the enhancement proposal.

The push command doesn't work in 0.6.1. It looks like the required ruby file isn't in the gemspec.

I've found out that with git there is little hope to work efficiently without knowing what happens under the hood (at least not for a long time). Could you add a section how does braid work on the high level?

Braid was only designed to work when the current directory is the root of the working tree. Running Braid in a subdirectory currently doesn't work (Braid looks for .braids.json in the subdirectory), and #62 will change Braid to raise an error in this case.

This issue is to support running in a subdirectory properly by looking for .braids.json in the root of the working tree and interpreting mirror paths relative to the root. As discussed in #44, command-line arguments should typically be interpreted relative to the current directory so that the shell's default path completion behavior works.

I'm wondering if it's possible to push changes using braid push to a different branch (that is not in the .braids file)?

The use case is to submit a pull-request which plays nicely with code reviews.

I think of using braid instead of submodules because it seems to be very helpful when using third party repos or subrepos.

Two things came in mind and I'd like to ask whether it's possible in future versions:

1. It would be very helpful to update the subtree to a tag instead of a hash, because the subrepo gets tagged when a stable state was reached.

2. It would be very helpful if I saw the hash (or tag) when invoking , so I don't need to open .braids or consult the git log.

When trying to use braid to manage a respository with its own submodules git cannot update the dependency's submodules.

E.g.

$ braid.sh add https://github.com/Valloric/YouCompleteMe
$ git submodule update --init --recursive
fatal: no submodule mapping found in .gitmodules for path 'YouCompleteMe/third_party/requests-futures'

Braid needs to find any .gitmodule files in the repository, recursively start tracking them, and check them out into the right location.

On update it would also need to refuse to change version for anything grandfathered in, and recursively get the appropriate versions based on the submodule requirements of the new parent version.

If the remote path for a mirror points to a single file, Braid should create a mirror consisting of the single file at the specified local path.

The use case that came up for me today: applying a gitignore file from https://github.com/github/gitignore to one's own repository. (The GitHub new repository wizard has an option to copy the gitignore file. The coup would be if I can get GitHub to add a checkbox to create .braids.json; it would be a great way to advertise a tool that I think a lot more people could benefit from using.)

This is mostly a matter of changing all the spots in the code that assume that Mirror::versioned_path returns a tree to also handle blobs. (Arguably, it's unfortunate that this is a special case at all, but that's beside the point.) The only interesting issue I see is what filenames braid diff should output. The current code that calls git diff --relative=MIRROR_PATH produces just a/ and b/, which makes sense in a way but might break tools. It's probably reasonable to use the basename of the file, but the upstream or downstream basename? They might differ, as in the gitignore use case above. [Thoughts, @realityforge?] Edit: My proposal: use a/UPSTREAM_NAME and b/DOWNSTREAM_NAME.

The change to support braiding of a subdirectory (#52) changed determine_target_revision to return a tree instead of a commit, even when the remote_path is empty. But the add and update commands in full-history mode still try to merge this object, which fails. Repro script for add:

git init
git config user.name 'Test user'
git config user.email '[email protected]'
touch dummy
git add dummy
git commit -m initial
braid add --full https://github.com/cristibalan/braid/ braid

Output:

Initialized empty Git repository in /home/user/objsheets/tools/braid/test-repo/.git/
[master (root-commit) 28f646e] initial
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 dummy
Braid: Adding mirror of 'https://github.com/cristibalan/braid'.
Braid: Resetting to '28f646e'.
Braid: Shell error: error: d9180f39cceb5ff05c122ac8326afbf27980fe0e: expected commit type, but the object dereferences to tree type

At a minimum, one would like full-history mirrors with empty remote_path to work as they did before. I'm not sure if there's a reasonable way to implement full-history mirrors of a subdirectory.

We should probably add a test for full-history mode or otherwise remove the feature. (Not to mention a test with a nonempty remote_path.)

I tried to step back to an earlier commit with "braid update --revision" but once I used "braid update --head" I cannot lock it to older revisions or I could not find out how (squashed = true).

The message I get (1.0.1):
Braid: Mirror 'sub' is already up to date.

Is this possible somehow or restricted intentionally?

When I run the test suite in my VM with global user.name and user.email unset, the push tests fail. I see these settings were added to the .travis.yml. We could document that these settings have to be set in order to run the test suite locally (which would mean I'd do my Braid development in a different VM) or perhaps find another workaround. But I wonder if having braid push use the global settings might be a poor design in the first place. What does the user expect? Should we use the author settings from the repository in which braid push was invoked (which would be a matter of copying the settings to the temporary repository)? For that matter, I can probably rewrite braid push so it generates a temporary commit in the original repository without touching the working tree (hooray for git plumbing commands), and that would also solve the problem.

There are a few broken and outdated links on the braid project page:

• github repo description link
• README: braid project on lighthouse link
• README: braid-gem google group link

When running braid update:

F, [2010-11-23T16:15:13.786780 #6609] FATAL -- : wrong number of arguments (1 for 0) (ArgumentError)
/Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/main-4.3.0/lib/main/parameter.rb:553:in block in fattr_block_for' /Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/fattr-2.2.0/lib/fattr.rb:90:ininstance_eval'
/Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/fattr-2.2.0/lib/fattr.rb:90:in call' /Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/fattr-2.2.0/lib/fattr.rb:90:inblock (2 levels) in fattrs'
/Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/fattr-2.2.0/lib/fattr.rb:139:in call' /Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/fattr-2.2.0/lib/fattr.rb:139:inblock (3 levels) in fattrs'
(eval):4:in verbose' /Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/braid-0.6.2/bin/braid:81:inblock (3 levels) in <top (required)>'
/Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/main-4.3.0/lib/main/program/class_methods.rb:147:in block in run' /Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/main-4.3.0/lib/main/program/class_methods.rb:136:incatch'
/Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/main-4.3.0/lib/main/program/class_methods.rb:136:in run' /Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/main-4.3.0/lib/main/factories.rb:18:inrun'
/Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/main-4.3.0/lib/main/factories.rb:25:in Main' /Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/gems/braid-0.6.2/bin/braid:13:in<top (required)>'
/Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/bin/braid:19:in load' /Users/dmcinnes/.rvm/gems/ruby-1.9.2-p0@webyp/bin/braid:19:in

braid diff silently ignores uncommitted changes. It should at least give a warning or error, or better, include them in the diff.

(Going ahead and filing this, though I may be the one to implement it anyway.)

A while ago, I locked several mirrors to particular revisions, which Braid then represented using the lock field. e6535aa changed the way a revision lock is represented and didn't keep support for reading the old format. So Braid would now update these mirrors to the latest commit on the branch if I let it. Was this intentional? I guess the damage may already be done for many users, but we could add the proper code now to read the old format. We need to get better at compatibility if we want people to take Braid seriously.

The cannot-guess-type message should be more informative.

It should tell the user that s/he can tell braid the type of mirror via the command-line option --type, in case braid cannot guess it.

There should be a unique line of text used in the gemspec, in the README, for display on GitHub, and so forth. Who has a good suggestion?

/cc @cristibalan @realityforge + @mislav

I've got the mentioned error message while trying to push changes to a repo .I looked a bit into the code and first removed the error message splitting in operations.rb:19 to get a full message. This outcropped the additional messge patch does not apply which led me to the stackoverflow thread http://stackoverflow.com/questions/4770177/git-patch-does-not-apply where somebody suggests o call git apply with the parameters --ignore-space-change --ignore-whitespace.
So I replaced --whitespace=nowarn with these parameters and was able to push without problems. I also tried the mentioned --whitespace=fix but this seems to replace all CRLF with LF.

Long story short, I would appreciate if the whitespace problem could be fixed in any way. Always getting the full error messages and not only the first line would be nice, too.

P.S.:
Thanks for a nice tool! Braid is the best and easiest submodule alternative i've seen so far.

#47 introduced a performance problem: re-fetching a remote from the cache after deleting and re-adding it can take a nontrivial amount of time even if all the objects are still present in the repository. For example, re-fetching a mirror of a subdirectory of DefinitelyTyped takes 10 seconds on my laptop. It looks like git fetch validates twice(!) that the receiving repository contains all objects that are transitively referenced by the refs being fetched but not by the refs already in the receiving repository (first time, second time), and each scan takes 5 seconds in my example. I haven't researched why git does this.

Depending on how severely the problem affects common Braid use cases (see #69), we should consider asking for a way to alter git's behavior or implementing a better workaround in Braid than manually passing --keep to every command. I have a bunch of ideas that I can write up if it becomes relevant.

My repo was not clean and I stashed it. Couple of file are untracked.

When I issue the command:

braid add git://github.com/fnichol/chef-rvm.git cookbooks/rvm

It says, Braid: Error: local changes are present

And its weird. Do I have to remove or commit the untracked files as well? I think it should have been worked though I've untracked files, right?

Hello,

I tried to install render_component (http://github.com/ewildgoose/render_component/tree/rails-2.3) using that command:

braid add git://github.com/ewildgoose/render_component.git --rails_plugin --branch=rails-2.3

Apparently that works, but braid is cloning the master branch instead of branch rails-2.3. On .braids file thats show the branch: rails-2.3 but this doesn't working correctly.

I tried some repositories and i having the same issue. Check that please...

My environment: Rails 2.3, Braid 0.5, Git 1.6.3.1, Debian Lenny

Each time a new version of Braid changes the .braids.json format in a way that older versions of Braid will process incorrectly (e.g., e6535aa), teams can get incorrect behavior unless they manually communicate and make sure that everyone who uses Braid upgrades it at the same time. We should add a version number to the configuration file and add a check to Braid that the version number is not newer than the version it understands.

Then, ideally, we'd make a one-time change to the configuration file to block out all versions of Braid from before the version check was added. Unfortunately, if we just rename it, then old versions of Braid will behave as if no mirrors are defined; some commands will fail with "mirror does not exist", but braid add will happily create a new configuration file, which will just lead to more confusion. The only way to reliably cause old versions of Braid to fail is to deliberately make .braids.json invalid JSON, but that's a pain for anyone who wants to process it with other tools. And if we wanted to block out versions of Braid from before the .braids.json name was introduced, we'd have to create a .braids file. This becomes ugly. Maybe the most reasonable thing is to forget about this and just tell teams to upgrade everyone to at least Braid 1.0.22, and then after that point, they will be notified when they need to upgrade.

On Windows, Braid writes .braids.json with LF line endings. Using Git for Windows with the recommended settings (core.autocrlf = true and core.safecrlf = warn), this produces a warning on git add:

warning: LF will be replaced by CRLF in .braids.json.
The file will have its original line endings in your working directory.

This often goes unnoticed because when Braid calls git add, it swallows the warning (arguably poor design). However, depending on how quickly Braid runs, the index entry for .braids.json may be left "racily clean", in which case the next git add command will print the warning again. This appears to be the cause of the intermittent warnings when I run the test suite on Windows.

Arguably git should be enhanced so that "racily clean" entries do not lead to intermittent duplicate safecrlf warnings (edit: thread). But I'd also argue that we should consider fixing the root cause by changing Braid to write the file with CRLF endings, at least if core.autocrlf = true.

We could look to the .gitmodules file for precedent. Interestingly, git submodule add also writes it with LF endings under Git for Windows and shows a warning. I inquired about this on the Git for Windows discussion group.

Hi @cristibalan,

Would you be able to activate TravisCI for this project? All you should need to do is visit https://travis-ci.org/profile/cristibalan and toggle the flag this project.

Thanks!

It appears that braid is not windows friendly. If that is the case and there is no plan to get working that is cool but please put a note on the front page.

If it is supposed to work let me know what information you need to debug this issue when running braid list in the root of my project.

A little history:
My boss setup the project and started using braid to handle the deps, it works really really well, but he is using ubuntu. I would like to add a new dep but this bug is currently stopping me.