Commit 08c7b06b authored by Daniel Paul Searles's avatar Daniel Paul Searles

Move remaining secure analyzers to core

Why:

* https://gitlab.com/groups/gitlab-org/-/epics/2098

This change addresses the need by:

* Remove required check for sast feature in CI template
* Remove required check for secret feature in CI template
* Document when analyzers were moved to core
parent 20c10042
---
title: Bring SAST to Core - bandit, Flawfinder, Gitleaks, Gosec, Kubesec, NodeJsScan, phpcs-security-audit, PMD, Security Code Scan, Sobelow, SpotBugs
merge_request: 37648
author:
type: changed
This diff is collapsed.
...@@ -19,7 +19,7 @@ malicious users to gain access to resources like deployment environments. ...@@ -19,7 +19,7 @@ malicious users to gain access to resources like deployment environments.
GitLab 11.9 includes a new check called Secret Detection. It scans the content of the repository GitLab 11.9 includes a new check called Secret Detection. It scans the content of the repository
to find API keys and other information that should not be there. to find API keys and other information that should not be there.
GitLab displays identified secrets as part of the SAST reports visibly in a few places: GitLab displays identified secrets visibly in a few places:
- [Security Dashboard](../security_dashboard/) - [Security Dashboard](../security_dashboard/)
- Pipelines' **Security** tab - Pipelines' **Security** tab
...@@ -46,6 +46,25 @@ CAUTION: **Caution:** ...@@ -46,6 +46,25 @@ CAUTION: **Caution:**
If you use your own Runners, make sure the Docker version installed If you use your own Runners, make sure the Docker version installed
is **not** `19.03.0`. See [troubleshooting information](../sast#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details. is **not** `19.03.0`. See [troubleshooting information](../sast#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details.
### Making Secret Detection available to all GitLab tiers
To make Secret Detection available to as many customers as possible, we have enabled it for all GitLab tiers.
However not all features are available on every tier. See the breakdown below for more details.
#### Summary of features per tier
Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/),
as shown in the following table:
| Capability | In Core | In Ultimate |
|:--------------------------------------------------------------------------|:--------------------|:-------------------|
| [Configure Secret Detection Scanners](#configuration) | **{check-circle}** | **{check-circle}** |
| [Customize Secret Detection Settings](#customizing-settings) | **{check-circle}** | **{check-circle}** |
| View [JSON Report](../sast/index.md#reports-json-format) | **{check-circle}** | **{check-circle}** |
| [Presentation of JSON Report in Merge Request](#overview) | **{dotted-circle}** | **{check-circle}** |
| [Interaction with Vulnerabilities](../vulnerabilities/index.md) | **{dotted-circle}** | **{check-circle}** |
| [Access to Security Dashboard](../security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** |
## Configuration ## Configuration
NOTE: **Note:** NOTE: **Note:**
......
...@@ -21,28 +21,6 @@ RSpec.describe 'SAST.gitlab-ci.yml' do ...@@ -21,28 +21,6 @@ RSpec.describe 'SAST.gitlab-ci.yml' do
end end
context 'when project has no license' do context 'when project has no license' do
it 'includes no jobs' do
expect { pipeline }.to raise_error(Ci::CreatePipelineService::CreateError)
end
end
context 'when project has Ultimate license' do
let(:license) { create(:license, plan: License::ULTIMATE_PLAN) }
before do
allow(License).to receive(:current).and_return(license)
end
context 'when SAST_DISABLE_DIND=false' do
before do
create(:ci_variable, project: project, key: 'SAST_DISABLE_DIND', value: 'false')
end
it 'includes orchestrator job' do
expect(build_names).to match_array(%w[sast])
end
end
context 'when SAST_DISABLED=1' do context 'when SAST_DISABLED=1' do
before do before do
create(:ci_variable, project: project, key: 'SAST_DISABLED', value: '1') create(:ci_variable, project: project, key: 'SAST_DISABLED', value: '1')
...@@ -96,5 +74,23 @@ RSpec.describe 'SAST.gitlab-ci.yml' do ...@@ -96,5 +74,23 @@ RSpec.describe 'SAST.gitlab-ci.yml' do
end end
end end
end end
context 'when project has Ultimate license' do
let(:license) { create(:license, plan: License::ULTIMATE_PLAN) }
before do
allow(License).to receive(:current).and_return(license)
end
context 'when SAST_DISABLE_DIND=false' do
before do
create(:ci_variable, project: project, key: 'SAST_DISABLE_DIND', value: 'false')
end
it 'includes orchestrator job' do
expect(build_names).to match_array(%w[sast])
end
end
end
end end
end end
...@@ -52,8 +52,7 @@ sast: ...@@ -52,8 +52,7 @@ sast:
rules: rules:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH
$GITLAB_FEATURES =~ /\bsast\b/
script: script:
- /analyzer run - /analyzer run
...@@ -65,7 +64,6 @@ bandit-sast: ...@@ -65,7 +64,6 @@ bandit-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /bandit/ $SAST_DEFAULT_ANALYZERS =~ /bandit/
exists: exists:
- '**/*.py' - '**/*.py'
...@@ -106,7 +104,6 @@ flawfinder-sast: ...@@ -106,7 +104,6 @@ flawfinder-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /flawfinder/ $SAST_DEFAULT_ANALYZERS =~ /flawfinder/
exists: exists:
- '**/*.c' - '**/*.c'
...@@ -120,7 +117,6 @@ kubesec-sast: ...@@ -120,7 +117,6 @@ kubesec-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /kubesec/ && $SAST_DEFAULT_ANALYZERS =~ /kubesec/ &&
$SCAN_KUBERNETES_MANIFESTS == 'true' $SCAN_KUBERNETES_MANIFESTS == 'true'
...@@ -132,7 +128,6 @@ gosec-sast: ...@@ -132,7 +128,6 @@ gosec-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /gosec/ $SAST_DEFAULT_ANALYZERS =~ /gosec/
exists: exists:
- '**/*.go' - '**/*.go'
...@@ -145,7 +140,6 @@ nodejs-scan-sast: ...@@ -145,7 +140,6 @@ nodejs-scan-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /nodejs-scan/ $SAST_DEFAULT_ANALYZERS =~ /nodejs-scan/
exists: exists:
- 'package.json' - 'package.json'
...@@ -158,7 +152,6 @@ phpcs-security-audit-sast: ...@@ -158,7 +152,6 @@ phpcs-security-audit-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /phpcs-security-audit/ $SAST_DEFAULT_ANALYZERS =~ /phpcs-security-audit/
exists: exists:
- '**/*.php' - '**/*.php'
...@@ -171,7 +164,6 @@ pmd-apex-sast: ...@@ -171,7 +164,6 @@ pmd-apex-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /pmd-apex/ $SAST_DEFAULT_ANALYZERS =~ /pmd-apex/
exists: exists:
- '**/*.cls' - '**/*.cls'
...@@ -184,7 +176,6 @@ secrets-sast: ...@@ -184,7 +176,6 @@ secrets-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /secrets/ $SAST_DEFAULT_ANALYZERS =~ /secrets/
security-code-scan-sast: security-code-scan-sast:
...@@ -195,7 +186,6 @@ security-code-scan-sast: ...@@ -195,7 +186,6 @@ security-code-scan-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /security-code-scan/ $SAST_DEFAULT_ANALYZERS =~ /security-code-scan/
exists: exists:
- '**/*.csproj' - '**/*.csproj'
...@@ -209,7 +199,6 @@ sobelow-sast: ...@@ -209,7 +199,6 @@ sobelow-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /sobelow/ $SAST_DEFAULT_ANALYZERS =~ /sobelow/
exists: exists:
- 'mix.exs' - 'mix.exs'
...@@ -222,7 +211,6 @@ spotbugs-sast: ...@@ -222,7 +211,6 @@ spotbugs-sast:
- if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false' - if: $SAST_DISABLED || $SAST_DISABLE_DIND == 'false'
when: never when: never
- if: $CI_COMMIT_BRANCH && - if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast\b/ &&
$SAST_DEFAULT_ANALYZERS =~ /spotbugs/ $SAST_DEFAULT_ANALYZERS =~ /spotbugs/
exists: exists:
- '**/*.groovy' - '**/*.groovy'
......
...@@ -21,8 +21,7 @@ secret_detection_default_branch: ...@@ -21,8 +21,7 @@ secret_detection_default_branch:
rules: rules:
- if: $SECRET_DETECTION_DISABLED - if: $SECRET_DETECTION_DISABLED
when: never when: never
- if: $CI_DEFAULT_BRANCH == $CI_COMMIT_BRANCH && - if: $CI_DEFAULT_BRANCH == $CI_COMMIT_BRANCH
$GITLAB_FEATURES =~ /\bsecret_detection\b/
script: script:
- /analyzer run - /analyzer run
...@@ -31,8 +30,7 @@ secret_detection: ...@@ -31,8 +30,7 @@ secret_detection:
rules: rules:
- if: $SECRET_DETECTION_DISABLED - if: $SECRET_DETECTION_DISABLED
when: never when: never
- if: $CI_COMMIT_BRANCH && $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH && - if: $CI_COMMIT_BRANCH && $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
$GITLAB_FEATURES =~ /\bsecret_detection\b/
script: script:
- git fetch origin $CI_DEFAULT_BRANCH $CI_BUILD_REF_NAME - git fetch origin $CI_DEFAULT_BRANCH $CI_BUILD_REF_NAME
- export SECRET_DETECTION_COMMIT_TO=$(git log --left-right --cherry-pick --pretty=format:"%H" refs/remotes/origin/$CI_DEFAULT_BRANCH...refs/remotes/origin/$CI_BUILD_REF_NAME | tail -n 1) - export SECRET_DETECTION_COMMIT_TO=$(git log --left-right --cherry-pick --pretty=format:"%H" refs/remotes/origin/$CI_DEFAULT_BRANCH...refs/remotes/origin/$CI_BUILD_REF_NAME | tail -n 1)
......
...@@ -512,7 +512,7 @@ RSpec.describe Ci::CreatePipelineService do ...@@ -512,7 +512,7 @@ RSpec.describe Ci::CreatePipelineService do
it 'pull it from Auto-DevOps' do it 'pull it from Auto-DevOps' do
pipeline = execute_service pipeline = execute_service
expect(pipeline).to be_auto_devops_source expect(pipeline).to be_auto_devops_source
expect(pipeline.builds.map(&:name)).to match_array(%w[build code_quality eslint-sast test]) expect(pipeline.builds.map(&:name)).to match_array(%w[build code_quality eslint-sast secret_detection_default_branch secrets-sast test])
end end
end end
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment