Signed-off-by: Erik Hollensbe <firstname.lastname@example.org>
|1 year ago|
|box-builds||1 year ago|
|cmd||1 year ago|
|git||1 year ago|
|gp||1 year ago|
|info||1 year ago|
|rq||1 year ago|
|.gitignore||1 year ago|
|.golangci.yml||1 year ago|
|.wwhrd.yml||1 year ago|
|LICENSE||1 year ago|
|Makefile||1 year ago|
|README.md||1 year ago|
|go.mod||1 year ago|
|go.sum||1 year ago|
|todo.txt||1 year ago|
gp: a proxy for remote git services. Make git fast.
gp allows you to localize many clones of the same SHA by way of marshalling git's operations through a queue. The idea is principally for CI/CD solutions which may need to make many full clones of a large remote git repository.
gp, there is a GRPC client and commandline tool,
gc, which can
trigger the proxy to cache a certain SHA if it exists. From there, you can
clone anew, or fetch an existing repository with the remote pointed at any git
server and fetch the contents.
gp only makes one promise at this point, that it'll never run git twice at
the same time for the same repository, avoiding corruption.
gp writes off the base directory (provided by the -d argument, but is
/tmp/git in the binary, and
/var/lib/git in the container by default). This
means if you start
git daemon, line it up with this directory so you can just
clone off of it.
$ docker run -itd --name gp-demo -p 127.0.0.1:9418:9418 erikh/gp # feel free to replace with your own repos. $ docker exec -it gp-demo gc trigger \ https://github.com/tinyci/ci-agents b0e39d2cbd0b95d1cb79b7760cafc1d71358da61 $ git clone git://127.0.0.1/tinyci/ci-agents $ cd ci-agents && git reflog show b0e39d2cbd0b95d1cb79b7760cafc1d71358da61
Current state of development
This is at the "proven concept" phase. A similar solution has been deployed at a company to great success. However, this specific code has not been tested very hard. You may run into issues specifically with long-term storage of the cache, or potentially (but unlikely) with force-pushes and time of use.
Always check that you got your SHA in your clone. We simply cannot guarantee the atomicity of the cache. Others may update or replace it as it changes upstream.
RAM is a big deal with large Git repositories. If you plan to use it with large
packs or a lot of heavily compressed objects, you may want to check your
resource limits before blaming
gp, which spawns
git with impunity.
gp does is run
git, it should be trivial to inject automated auth
of any kind into a container with a
$HOME directory set properly in the
environment and a
.gitconfig to provide the credentials. However, as of this
writing I have not tried this.
- It's fast enough, but it probably shouldn't shell out to git.
- It doesn't have queue limits.
- It doesn't have auth.
- It doesn't have https or SSH support in the container.
Erik Hollensbe email@example.com
MIT (c) 2020