Git Product home page Git Product logo

gitprotocolio's Introduction

gitprotocolio

A Git protocol parser written in Go.

This is more like an experimental project to better understand the protocol.

This is not an official Google product (i.e. a 20% project).

Background

Git protocol is defined in Documentation/technical/pack-protocol.txt. This is not a complete definition. Also, a transport specific spec Documentation/technical/http-protocol.txt is not complete. This project was started so that these upstream protocol spec becomes more accurate. To verify the written syntax is accurate, this project includes a Git protocol parser written in Go, and have end-to-end test suites.

This makes it easy to write a test case for Git client. Currently the test cases are run against the canonical Git implementation, but this can be extended to run against JGit, etc.. Also it makes it easy to test an attack case. With this library, one can write an attack case like git-bomb against Git protocol by producing a request that is usually not produced by a sane Git client. Protocol properties can also be checked. For example, it's possible to write a test to check valid request/response's prefixes are not a valid request/response. This property makes sure a client won't process an incomplete response thinking it's complete.

TODOs

  • Protocol semantics is not defined.

    The syntax is relatively complete. The semantics is not even mentioned. One idea is to define the semantics by treating the request/response as an operation to modify Git repositories. This perspective makes it possible to define a formal protocol semantics in a same way as programming language formal semantics.

    Defining a simple git-push semantics seems easy. Defining a pack negotiation semantics for shallow cloned repositories seems difficult.

  • Upstream pack-protocol.txt is not updated.

    The initial purpose, create a complete pack-protocol.txt, is not yet done. We can start from a minor fix (e.g. capability separator in some places is space not NUL). Also relationship between Git protocol and network transports (HTTPS, SSH, Git wire) are good to be mentioned.

  • Bidi-transports are not tested and defined.

    Git's bidi-transports, SSH and Git-wire protocol, are not tested with this project and the protocol syntax is also not defined. The majority of the syntax is same, but there's a slight difference. Go has an SSH library, so it's easy to run a test SSH server.

gitprotocolio's People

Contributors

draftcode avatar sluongng avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gitprotocolio's Issues

Project status?

Hi,

I am reviewing the implementation here for my personal project and I think there are still some unfinished pieces.

The spec for V2 were also added some new functionality in recent months.

I was wondering if PR are accepted or would you prefer others maintaining a fork? Or perhaps contributing this to https://github.com/go-git/go-git which is still missing a Protocol V2 implementation?

Unable to parse git-upload-pack payload

How to reproduce

# Setup server
git clone https://github.com/sluongng/git-cache/
git checkout 07f1136c7d09c8281e4e0d8a8b96e3816dffbdac
go run .

# Setup client
# This should clone git.git from Github through the simple golang proxy
git -c http.extraHeader='GIT-CACHE-UPSTREAM: https://github.com' clone -vvv http://127.0.0.1:8080/git/git.git

Expected result

The middleware layer in sluongng/git-cache@07f1136#diff-0eb779b9e49d8e44b0f36923fdb8d87d5ee024f886eefc45deec4ec88380a087R56 should be able to parse the request body into ProtocolV2RequestChunk and serialize the Chunks as JSON and log them out.

Actual result

ls-ref command seems to work fine, but the one next git-upload-pack command failed to be parse by gitprotocolio

Unable to scan request: strconv.ParseUint: parsing "\x1f\x8b\b\x00": invalid syntax

Which most likely coming from

sz, err := strconv.ParseUint(string(data[:4]), 16, 32)

The vision is not clear

In order to contribute, having clear vision help us to specify missions.

Could you please explain more the vision of this repository?
i.e: audience ... and so on.

Git Protocol Version not clear

There seems to be both Protocol V1 and V2 files in the codebase. Is the target of the repo to support both protocol versions or just focus on the V2 highlighted in googleblog a few weeks ago? Regardless, better clarity would be nice in the readme.

Additionally what are some of the future tasks? I would love to be a contributor.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.