Commit 20f5f34b authored by Evan Read's avatar Evan Read

Merge branch 'docs-artifacts-delete-invalid-references' into 'master'

Docs - Artifacts - delete invalid references

See merge request gitlab-org/gitlab!62592
parents fc259489 0d54bc56
......@@ -128,6 +128,11 @@ This command permanently deletes the log files and is irreversible.
find /var/opt/gitlab/gitlab-rails/shared/artifacts -name "job.log" -mtime +60 -delete
```
NOTE:
After execution, broken file references can be reported when running
[`sudo gitlab-rake gitlab:artifacts:check`](raketasks/check.md#uploaded-files-integrity).
For more information, see [delete references to missing artifacts](raketasks/check.md#delete-references-to-missing-artifacts).
## Incremental logging architecture
> - [Deployed behind a feature flag](../user/feature_flags.md), disabled by default.
......
......@@ -246,6 +246,41 @@ end
p "#{uploads_deleted} remote objects were destroyed."
```
### Delete references to missing artifacts
`gitlab-rake gitlab:artifacts:check VERBOSE=1` detects when artifacts (or `job.log` files):
- Are deleted outside of GitLab.
- Have references still in the GitLab database.
When this scenario is detected, the Rake task displays an error message. For example:
```shell
Checking integrity of Job artifacts
- 3..8: Failures: 2
- Job artifact: 3: #<Errno::ENOENT: No such file or directory @ rb_sysopen - /var/opt/gitlab/gitlab-rails/shared/artifacts/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce/2021_05_26/5/3/job.log>
- Job artifact: 8: #<Errno::ENOENT: No such file or directory @ rb_sysopen - /var/opt/gitlab/gitlab-rails/shared/artifacts/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce/2021_05_26/6/8/job.log>
Done!
```
To delete these references to missing local artifacts (`job.log` files):
1. Open the [GitLab Rails Console](../operations/rails_console.md#starting-a-rails-console-session).
1. Run the following Ruby code:
```ruby
artifacts_deleted = 0
::Ci::JobArtifact.all.each do |artifact| ### Iterate artifacts
# next if artifact.file.filename != "job.log" ### Uncomment if only `job.log` files' references are to be processed
next if artifact.file.exists? ### Skip if the file reference is valid
artifacts_deleted += 1
puts "#{artifact.id} #{artifact.file.path} is missing." ### Allow verification before destroy
# artifact.destroy! ### Uncomment to actually destroy
end
puts "Count of identified/destroyed invalid references: #{artifacts_deleted}"
```
### Delete references to missing LFS objects
If `gitlab-rake gitlab:lfs:check VERBOSE=1` detects LFS objects that exist in the database
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment