[[Image(git-bigfiles.png, right, nolink)]]
= git-bigfiles - Git for big files =

'''git-bigfiles''' is our fork of [http://git-scm.com/ Git]. It has two goals:

 * make life bearable for people using Git on projects hosting very large files (hundreds of megabytes)
 * merge back as many changes as possible into upstream Git once they’re of acceptable quality

== Features ==

'''git-bigfiles''' already features the following fixes:

 * '''git-config''': add `core.bigFileThreshold` option to treat files larger than this value as big files
 * '''git-p4''': fix poor string building performance when importing big files (git-p4 is now only marginally faster on Linux but 4 to 10 times faster on Win32)
 * '''git-p4''': do not perform keyword replacements on big files
 * '''git-fast-import''': avoid computing deltas on big files and deflate them on-the-fly (fast-import is now three times as fast and uses 5 times less memory with big files)
 * '''git-pack-objects''': do not perform delta computation on big files (pack-objects is now 10% faster, but there is no memory gain yet)

== Results ==

Git `fast-import` memory usage and running time when importing a repository with binary files up to 150 MiB, then `git-gc` on the same repository:

[[Image(fast-import-memory.png, 200px)]]
[[Image(pack-objects-memory.png, 169px)]]

== Development ==

git-bigfiles development is centralised in a `repo.or.cz` Git repository.

Clone the repository:

{{{
git clone git://repo.or.cz/git/git-bigfiles.git
}}}

If you already have a working copy of upstream git, you may save a lot of bandwidth by doing:

{{{
git clone git://repo.or.cz/git/git-bigfiles.git --reference /path/to/git/
}}}

The main Git repository is constantly merged into git-bigfiles. See the [http://repo.or.cz/w/git/git-bigfiles.git git-bigfiles] repository on [http://repo.or.cz/ repo.or.cz].

== To do ==

The following commands are real memory hogs with big files:

 * `git diff-tree --stat --summary -M <sha1> HEAD`
 * `git gc` (as called by `git merge`) fails with ''warning: suboptimal pack - out of memory''

== Readings ==

 * 2006/02/08: Linus on [http://kerneltrap.org/mailarchive/git/2006/2/8/200594 Handling large files with GIT]
 * 2006/07/10: [http://kerneltrap.org/mailarchive/git/2006/7/10/208528 Revisiting large binary files issue]
 * 2008/09/09: [http://www.mail-archive.com/git-users@googlegroups.com/msg00024.html malloc fails when dealing with huge files]