You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Erik Hollensbe 2fa9d32a20 initial commit 1 year ago
box-builds initial commit 1 year ago
cmd initial commit 1 year ago
git initial commit 1 year ago
gp initial commit 1 year ago
info initial commit 1 year ago
rq initial commit 1 year ago
.gitignore initial commit 1 year ago
.golangci.yml initial commit 1 year ago
.wwhrd.yml initial commit 1 year ago
LICENSE initial commit 1 year ago
Makefile initial commit 1 year ago initial commit 1 year ago
go.mod initial commit 1 year ago
go.sum initial commit 1 year ago
todo.txt initial commit 1 year ago

gp: a proxy for remote git services. Make git fast.

gp allows you to localize many clones of the same SHA by way of marshalling git's operations through a queue. The idea is principally for CI/CD solutions which may need to make many full clones of a large remote git repository.

To use gp, there is a GRPC client and commandline tool, gc, which can trigger the proxy to cache a certain SHA if it exists. From there, you can clone anew, or fetch an existing repository with the remote pointed at any git server and fetch the contents.

gp only makes one promise at this point, that it'll never run git twice at the same time for the same repository, avoiding corruption.

gp writes off the base directory (provided by the -d argument, but is /tmp/git in the binary, and /var/lib/git in the container by default). This means if you start git daemon, line it up with this directory so you can just clone off of it.


$ docker run -itd --name gp-demo -p erikh/gp
# feel free to replace with your own repos.
$ docker exec -it gp-demo gc trigger \ b0e39d2cbd0b95d1cb79b7760cafc1d71358da61
$ git clone git:// 
$ cd ci-agents && git reflog show b0e39d2cbd0b95d1cb79b7760cafc1d71358da61

Current state of development

This is at the "proven concept" phase. A similar solution has been deployed at a company to great success. However, this specific code has not been tested very hard. You may run into issues specifically with long-term storage of the cache, or potentially (but unlikely) with force-pushes and time of use.

Always check that you got your SHA in your clone. We simply cannot guarantee the atomicity of the cache. Others may update or replace it as it changes upstream.


RAM is a big deal with large Git repositories. If you plan to use it with large packs or a lot of heavily compressed objects, you may want to check your resource limits before blaming gp, which spawns git with impunity.

Proxy Authentication

Since all gp does is run git, it should be trivial to inject automated auth of any kind into a container with a $HOME directory set properly in the environment and a .gitconfig to provide the credentials. However, as of this writing I have not tried this.


  • It's fast enough, but it probably shouldn't shell out to git.
  • It doesn't have queue limits.
  • It doesn't have auth.
  • It doesn't have https or SSH support in the container.


Erik Hollensbe


MIT (c) 2020