Deemwar Monads & Abstractions

How big is a Git commit?

Source: How big is a Git commit?, by Dave Gauer

Changing one word in one file grew a small repo’s .git folder by 17,146 bytes. Almost none of that is your change. A commit writes a few small compressed objects, and the rest is folders, logs and bookkeeping.

17,146 B .git growth, bar → baz
4 new objects
463 B those objects on disk
16,384 B new folders, as du counts them

The puzzle

Dave Gauer measured .git with du -sb after each step of a small experiment. All objects stayed loose: no packfiles were created.

Step.git grows byNote
Empty git init64,828 Btotal size of a new .git
Commit 250 tiny files (5 dirs × 50, each “foo”)47,640 B112,468 B total
Edit one file, foo → bar17,145 B129,613 B total
Edit it again, bar → baz17,146 B146,759 B total
Same 3-byte edit in a one-file repo8,709 Bsmaller tree, less overhead

What one commit writes

Git stores snapshots, not diffs. Every object is named by the hash of its content, so Git writes only what is new and points at everything else by hash. For the bar → baz edit, Git wrote four objects, from the bottom up.

  1. 1

    Blob “baz”

    the new content of file 1/1

    The only object that holds your actual change.

  2. 2

    Tree for directory 1

    lists all 50 files

    One file changed, so the whole directory listing is rewritten with the new blob hash. The other 49 entries point at blobs that already exist.

  3. 3

    Root tree

    lists directories 1 to 5

    Points at the new tree for directory 1 and the four unchanged trees already stored.

  4. 4

    Commit

    root tree, parent, author, message

    Ties the snapshot to its history.

Filled boxes are written by this commit; dashed lines point at objects Git already has.

Content addressing also explains the first commit: all 250 files contain “foo”, so they share one blob. In our reproduction, three commits produced 12 objects in total.

Where the 17 KB went

The source’s du -b listing shows four new object folders at 4,253, 4,302, 4,115 and 4,177 bytes. An empty folder there counts as 4,096. Subtract it, and the four objects weigh 157 + 206 + 19 + 81 = 463 bytes.

Folders16,384 B
Logs150 B
Index, refs149 B
Objects463 B

Git spreads loose objects over fan-out folders named after the first two hex digits of the hash (.git/objects/81, fc, 76, 92). Each new folder costs 4,096 bytes as du counts directories, so 4 × 4,096 = 16,384 bytes of the growth is folders, not data.

Our reproduction

We repeated the experiment with git on macOS and read each object with git cat-file -s (raw size) and ls -l (compressed size on disk).

ObjectRawOn disk
Tree for directory 11,491 B206 B
Root tree140 B81 B
Blob “baz”4 B19 B
Commit165 B129 B (the source’s is 157 B; author lines differ)

After git gc, our whole three-commit repo packed to about 2 KiB (git count-objects -v reports size-pack: 2).

Compression does the heavy lifting

Git zlib-compresses every object, so large files cost less than their size. How much less depends on how much repetition they contain.

583,840 B binary.git grew by 297,873 B
16,415,223 B source file.git grew by 1,622,188 B

The source sums it up: for larger commits with few files, the data stored in .git is 50% to 10% of the uncompressed files.

Packfiles

Loose objects are not the end state. Git eventually packs them into packfiles, storing similar objects as deltas against each other. That removes the per-folder overhead and the redundancy between versions of the same file.

Takeaway

Try it

Read more

How big is a Git commit? by Dave Gauer (ratfactor), the source of the experiment.

Hacker News discussion.

Git docs: loose object format · Git internals: packfiles.

Julia Evans’ work on Git’s data model documentation (jvns.ca), credited by the source for identifying the objects.

A companion Short covers the same experiment; its link will be added here.

Sources: every .git size, du figure and compression number is from Dave Gauer’s card. Object sizes after subtraction are arithmetic on his du listing. Raw sizes, the commit’s 129 B, the 12 objects and the 2 KiB pack are our reproduction.