Merge branch '249838-api-fuzzing-with-postman' into 'master'

Resolve "API Fuzzing using a Postman Collection"

See merge request gitlab-org/gitlab!46476

doc/user/application_security/api_fuzzing/index.md

@@ -23,7 +23,10 @@ you can run fuzz tests as part your CI/CD workflow.
   - SOAP
   - GraphQL
   - Form bodies, JSON, or XML
-- An OpenAPI definition, or HTTP Archive (HAR) of requests to test
+- One of the following assets to provide APIs to test:
+  - OpenAPI v2 API definition
+  - HTTP Archive (HAR) of API requests to test
+  - Postman Collection v2.0 or v2.1
 
 ## When fuzzing scans run
 
@@ -48,15 +51,17 @@ changes, other pipelines, or other scanners) during a scan could cause inaccurat
 
 ## Configuration
 
-There are two ways to perform scans. See the configuration section for the one you wish to use:
+There are three ways to perform scans. See the configuration section for the one you wish to use:
 
 - [OpenAPI v2 specification](#openapi-specification)
 - [HTTP Archive (HAR)](#http-archive-har)
+- [Postman Collection v2.0 or v2.1](#postman-collection)
 
 Examples of both configurations can be found here:
 
 - [Example OpenAPI v2 specification project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/openapi)
 - [Example HTTP Archive (HAR) project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/har)
+- [Example Postman Collection project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing/postman-collection/)
 
 ### OpenAPI Specification
 
@@ -229,6 +234,97 @@ DANGER: **Warning:**
 the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting
 data. Only run fuzzing against a test server.
 
+### Postman Collection
+
+The [Postman API Client](https://www.postman.com/product/api-client/) is a popular tool that
+developers and testers use to call various types of APIs. The API definitions
+[can be exported as a Postman Collection file](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-postman-data)
+for use with API Fuzzing. When exporting, make sure to select a supported version of Postman
+Collection: v2.0 or v2.1.
+
+When used with GitLab's API fuzzer, Postman Collections must contain definitions of the web API to
+test with valid data. The API fuzzer extracts all the API definitions and uses them to perform
+testing.
+
+DANGER: **Warning:**
+Postman Collection files may contain sensitive information such as authentication tokens, API keys,
+and session cookies. We recommend that you review the Postman Collection file contents before adding
+them to a repository.
+
+Follow these steps to configure API fuzzing to use a Postman Collection file that provides
+information about the target API to test:
+
+1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate)
+   the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml)
+   that's provided as part of your GitLab installation. To do so, add the following to your
+   `.gitlab-ci.yml` file:
+
+   ```yaml
+   include:
+     - template: API-Fuzzing.gitlab-ci.yml
+   ```
+
+1. Add the configuration file [`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml)
+   to your repository's root as `.gitlab-api-fuzzing.yml`.
+
+1. The [configuration file](#configuration-files) has several testing profiles defined with varying
+   amounts of fuzzing. We recommend that you start with the `Quick-10` profile. Testing with this
+   profile completes quickly, allowing for easier configuration validation.
+
+   Provide the profile by adding the `FUZZAPI_PROFILE` variable to your `.gitlab-ci.yml` file,
+   substituting `Quick-10` for the profile you choose:
+
+   ```yaml
+   include:
+     - template: API-Fuzzing.gitlab-ci.yml
+
+   variables:
+     FUZZAPI_PROFILE: Quick-10
+   ```
+
+1. Add the `FUZZAPI_POSTMAN_COLLECTION` variable and set it to the Postman Collection's location:
+
+   ```yaml
+   include:
+     - template: API-Fuzzing.gitlab-ci.yml
+
+   variables:
+     FUZZAPI_PROFILE: Quick-10
+     FUZZAPI_POSTMAN_COLLECTION: postman-collection_serviceA.json
+   ```
+
+1. The target API instance's base URL is also required. Provide it by using the `FUZZAPI_TARGET_URL`
+   variable or an `environment_url.txt` file.
+
+   Adding the URL in an `environment_url.txt` file at your project's root is great for testing in
+   dynamic environments. To run API fuzzing against an app dynamically created during a GitLab CI/CD
+   pipeline, have the app persist its domain in an `environment_url.txt` file. API fuzzing
+   automatically parses that file to find its scan target. You can see an
+   [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml).
+
+   Here's an example of using `FUZZAPI_TARGET_URL`:
+
+   ```yaml
+   include:
+     - template: API-Fuzzing.gitlab-ci.yml
+
+   variables:
+     FUZZAPI_PROFILE: Quick-10
+     FUZZAPI_POSTMAN_COLLECTION: postman-collection_serviceA.json
+     FUZZAPI_TARGET_URL: http://test-deployment/
+   ```
+
+This is a minimal configuration for API Fuzzing. From here you can:
+
+- [Run your first scan](#running-your-first-scan).
+- [Add authentication](#authentication).
+- Learn how to [handle false positives](#handling-false-positives).
+
+DANGER: **Warning:**
+**NEVER** run fuzz testing against a production server. Not only can it perform *any* function that
+the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting
+data. Only run fuzzing against a test server.
+
 ### Authentication
 
 Authentication is handled by providing the authentication token as a header or cookie. You can
@@ -398,6 +494,7 @@ increases as the numbers go up. To use a configuration file, add it to your repo
 | `FUZZAPI_REPORT`            |Scan report filename. Defaults to `gl-api_fuzzing-report.xml`. |
 |[`FUZZAPI_OPENAPI`](#openapi-specification)|OpenAPI specification file or URL. |
 |[`FUZZAPI_HAR`](#http-archive-har)|HTTP Archive (HAR) file. |
+|[`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection)|Postman Collection file. |
 |[`FUZZAPI_OVERRIDES_FILE`](#overrides)     |Path to a JSON file containing overrides. |
 |[`FUZZAPI_OVERRIDES_ENV`](#overrides)      |JSON string containing headers to override. |
 |[`FUZZAPI_OVERRIDES_CMD`](#overrides)      |Overrides command. |

ee/changelogs/unreleased/249838-api-fuzzing-with-postman.yml

+---
+title: Update API Fuzzing template to support use of a Postman Collection
+merge_request: 46476
+author:
+type: added

ee/spec/lib/gitlab/ci/templates/api_fuzzing_gitlab_ci_yaml_spec.rb

@@ -60,8 +60,8 @@ RSpec.describe 'API-Fuzzing.gitlab-ci.yml' do
       end
 
       context 'by default' do
-        it 'includes no job' do
-          expect { pipeline }.to raise_error(Ci::CreatePipelineService::CreateError)
+        it 'includes a job' do
+          expect(build_names).to match_array(%w[apifuzzer_fuzz])
         end
       end
 
@@ -87,6 +87,17 @@ RSpec.describe 'API-Fuzzing.gitlab-ci.yml' do
         end
       end
 
+      context 'when configured with Postman' do
+        before do
+          create(:ci_variable, project: project, key: 'FUZZAPI_POSTMAN_COLLECTION', value: 'testing.json')
+          create(:ci_variable, project: project, key: 'FUZZAPI_TARGET_URL', value: 'http://example.com')
+        end
+
+        it 'includes job' do
+          expect(build_names).to match_array(%w[apifuzzer_fuzz])
+        end
+      end
+
       context 'when FUZZAPI_D_TARGET_IMAGE is present' do
         before do
           create(:ci_variable, project: project, key: 'FUZZAPI_D_TARGET_IMAGE', value: 'imagename:latest')

lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml

@@ -61,11 +61,17 @@ apifuzzer_fuzz:
         - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH &&
                 $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME
           when: never
-        - if: $FUZZAPI_HAR == null && $FUZZAPI_OPENAPI == null
-          when: never
         - if: $GITLAB_FEATURES =~ /\bapi_fuzzing\b/
     script:
         #
+        # Validate options
+        - |
+            if [ "$FUZZAPI_HAR$FUZZAPI_OPENAPI$FUZZAPI_POSTMAN_COLLECTION" == "" ]; then \
+                echo "Error: One of FUZZAPI_HAR, FUZZAPI_OPENAPI, or FUZZAPI_POSTMAN_COLLECTION must be provided."; \
+                echo "See https://docs.gitlab.com/ee/user/application_security/api_fuzzing/ for information on how to configure API Fuzzing."; \
+                exit 1; \
+            fi
+        #
         # Run user provided pre-script
         - sh -c "$FUZZAPI_PRE_SCRIPT"
         #
@@ -96,8 +102,6 @@ apifuzzer_fuzz_dnd:
         - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH &&
                 $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME
           when: never
-        - if: $FUZZAPI_HAR == null && $FUZZAPI_OPENAPI == null
-          when: never
         - if: $GITLAB_FEATURES =~ /\bapi_fuzzing\b/
     services:
         - docker:19.03.12-dind
@@ -142,7 +146,7 @@ apifuzzer_fuzz_dnd:
         # Start worker container if provided
         - |
             if [ "$FUZZAPI_D_WORKER_IMAGE" != "" ]; then \
-                echo "Starting worker image $FUZZAPI_D_WORKER_IMAGE" \
+                echo "Starting worker image $FUZZAPI_D_WORKER_IMAGE"; \
                 docker run \
                     --name worker \
                     --network $FUZZAPI_D_NETWORK \
@@ -153,6 +157,7 @@ apifuzzer_fuzz_dnd:
                     -e FUZZAPI_REPORT \
                     -e FUZZAPI_HAR \
                     -e FUZZAPI_OPENAPI \
+                    -e FUZZAPI_POSTMAN_COLLECTION \
                     -e FUZZAPI_TARGET_URL \
                     -e FUZZAPI_OVERRIDES_FILE \
                     -e FUZZAPI_OVERRIDES_ENV \
@@ -174,6 +179,11 @@ apifuzzer_fuzz_dnd:
         # Start API Fuzzing provided worker if no other worker present
         - |
             if [ "$FUZZAPI_D_WORKER_IMAGE" == "" ]; then \
+                if [ "$FUZZAPI_HAR$FUZZAPI_OPENAPI$FUZZAPI_POSTMAN_COLLECTION" == "" ]; then \
+                    echo "Error: One of FUZZAPI_HAR, FUZZAPI_OPENAPI, or FUZZAPI_POSTMAN_COLLECTION must be provided."; \
+                    echo "See https://docs.gitlab.com/ee/user/application_security/api_fuzzing/ for information on how to configure API Fuzzing."; \
+                    exit 1; \
+                fi; \
                 docker run \
                     --name worker \
                     --network $FUZZAPI_D_NETWORK \
@@ -185,6 +195,7 @@ apifuzzer_fuzz_dnd:
                     -e FUZZAPI_REPORT \
                     -e FUZZAPI_HAR \
                     -e FUZZAPI_OPENAPI \
+                    -e FUZZAPI_POSTMAN_COLLECTION \
                     -e FUZZAPI_TARGET_URL \
                     -e FUZZAPI_OVERRIDES_FILE \
                     -e FUZZAPI_OVERRIDES_ENV \