Summary

Files larger than 4 GB are not correctly handled on Windows.

For example:

• after cloning a file bigger than 4GB, the downloaded file does not appear to be complete.
• git checkout fails with Smudge error: Error downloading <object_name>

This is the case when using either Git core or Git LFS.

Environment

Windows Operating System

Bitbucket DC version 8.9.5

Diagnosis

The downloaded file after git clone operation does not appear to be complete.  (or)

The git checkout operation fails with an error

Smudge error: Error downloading <object_name> (d1XXXX): cannot write data to temporary file "D:\\Drive\\Git\\Windows_OS\\.git\\lfs\\incomplete\\c1bxxxxxxxxxxxxxxx": LFS: unexpected EOF

Enable git trace logging

For Windows:
set GIT_TRACE_PACKET=1
set GIT_TRACE=1 
set GIT_CURL_VERBOSE=1
set GIT_TRANSFER_TRACE=1

For Linux:
export GIT_TRACE_PACKET=1
export GIT_TRACE=1 
export GIT_CURL_VERBOSE=1
export GIT_TRANSFER_TRACE=1

and try to perform git checkout operation. The checkout operation fails with the below error messages

<DATE:TIMESTAMP> trace git-lfs: xfer: Attempting to resume download of "c1XXXXXXXXXXXX" from byte 1083359179
<DATE:TIMESTAMP> trace git-lfs: HTTP: GET <BITBUCKET_BASE_URL>/rest/git-lfs/storage/WINDOWSOS/c1XXXXXXXXXXXX
> GET /rest/git-lfs/storage/WINDOWSOS/c1XXXXXXXXXXXX HTTP/1.1
.
<DATE:TIMESTAMP> trace git-lfs: xfer: failed to resume download for "c1XXXXXXXXXXXX" from byte 1083359179: expected status code 206, received 200. Re-downloading from start
.
<DATE:TIMESTAMP> trace git-lfs: xfer: adapter "basic" worker 0 finished job for "c1XXXXXXXXXXXX"
<DATE:TIMESTAMP> trace git-lfs: tq: retrying object c1XXXXXXXXXXXX: cannot write data to temporary file "D:\\Drive\\Git\\Windows_OS\\.git\\lfs\\incomplete\\c1bxxxxxxxxxxxxxxx": LFS: unexpected EOF

Cause

This issue is caused by the following issue in the Git core project (which is not owned by the Bitbucket Server team):

• Commiting files larger than 4 GB

Workaround

When using Git LFS, smudging should be disabled completely. 

To give a brief overview of the smudge filter, large files are replaced with pointers with GIT LFS. These pointers are replaced by the actual files during git checkout process instead of clone, thus making the clones faster. The Git smudge filter is what converts the LFS pointer stored in Git with the actual large file from the LFS server. If the local repository does not have the LFS object, the smudge filter will automatically download it.

Disable smudging:

$ git lfs install --skip-smudge
$ git lfs env
... snip
git config filter.lfs.process = "git-lfs filter-process --skip"
git config filter.lfs.smudge = "git-lfs smudge --skip -- %f"

Run git lfs env command to verify if smudge filter is disabled.

$ git lfs env

git config filter.lfs.process = "git-lfs filter-process --skip"
git config filter.lfs.smudge = "git-lfs smudge --skip -- %f"

By disabling the smudge filter, the LFS objects are to be pulled manually.

Use the following git lfs command to explicitly pull changes in the branches or fetch updates from the remote:

git lfs pull

Warning added in Git LFS

The following message will be added to the Git client log when this issue is encountered:

Encountered <number> file(s) that may not have been copied correctly on Windows
git lfs help smudge` for more details

This has been introduced as part of this Git pull request.