Commit 973f599e authored by Russell Dickenson's avatar Russell Dickenson

Merge branch '284446-unassigned-pages-futuretense-4' into 'master'

Fix future tense in development pages

See merge request gitlab-org/gitlab!48197
parents bb5d25a9 8b215be5
......@@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
Whenever you add comment to the code that is expected to be addressed at any time
in future, please create a technical debt issue for it. Then put a link to it
to the code comment you've created. This will allow other developers to quickly
to the code comment you've created. This allows other developers to quickly
check if a comment is still relevant and what needs to be done to address it.
Examples:
......
......@@ -89,28 +89,28 @@ Go 1.12 introduced checksum databases and module proxies.
### Checksums
In addition to `go.mod`, a module will have a `go.sum` file. This file records a
In addition to `go.mod`, a module has a `go.sum` file. This file records a
SHA-256 checksum of the code and the `go.mod` file of every version of every
dependency that is referenced by the module or one of the module's dependencies.
Go continually updates `go.sum` as new dependencies are referenced.
When Go fetches the dependencies of a module, if those dependencies already have
an entry in `go.sum`, Go will verify the checksum of these dependencies. If the
checksum does not match what is in `go.sum`, the build will fail. This ensures
an entry in `go.sum`, Go verifies the checksum of these dependencies. If the
checksum does not match what is in `go.sum`, the build fails. This ensures
that a given version of a module cannot be changed by its developers or by a
malicious party without causing build failures.
Go 1.12+ can be configured to use a checksum database. If configured to do so,
when Go fetches a dependency and there is no corresponding entry in `go.sum`, Go
will query the configured checksum database(s) for the checksum of the
queries the configured checksum database(s) for the checksum of the
dependency instead of calculating it from the downloaded dependency. If the
dependency cannot be found in the checksum database, the build will fail. If the
dependency cannot be found in the checksum database, the build fails. If the
downloaded dependency's checksum does not match the result from the checksum
database, the build will fail. The following environment variables control this:
database, the build fails. The following environment variables control this:
- `GOSUMDB` identifies the name, and optionally the public key and server URL,
of the checksum database to query.
- A value of `off` will entirely disable checksum database queries.
- A value of `off` entirely disables checksum database queries.
- Go 1.13+ uses `sum.golang.org` if `GOSUMDB` is not defined.
- `GONOSUMDB` is a comma-separated list of module suffixes that checksum
database queries should be disabled for. Wildcards are supported.
......@@ -125,8 +125,8 @@ attempts to fetch the dependency from the configured proxies, in order. The
following environment variables control this:
- `GOPROXY` is a comma-separated list of module proxies to query.
- A value of `direct` will entirely disable module proxy queries.
- If the last entry in the list is `direct`, Go will fall back to the process
- A value of `direct` entirely disables module proxy queries.
- If the last entry in the list is `direct`, Go falls back to the process
described [above](#fetching-packages) if none of the proxies can provide the
dependency.
- Go 1.13+ uses `proxy.golang.org,direct` if `GOPROXY` is not defined.
......@@ -159,7 +159,7 @@ From Go 1.12 onward, the process for fetching a module or package is as follows:
The downloaded source must contain a `go.mod` file. The `go.mod` file must
contain a `module` directive that specifies the name of the module. If the
module name as specified by `go.mod` does not match the name that was used to
fetch the module, the module will fail to compile.
fetch the module, the module fails to compile.
If the module is being fetched directly and no version was specified, or if the
module is being added as a dependency and no version was specified, Go uses the
......@@ -172,9 +172,9 @@ latest that is also a valid semantic version.
In versions prior to Go 1.13, support for authenticating requests made by Go was
somewhat inconsistent. Go 1.13 improved support for `.netrc` authentication. If
a request is made over HTTPS and a matching `.netrc` entry can be found, Go will
add HTTP Basic authentication credentials to the request. Go will not
authenticate requests made over HTTP. Go will reject HTTP-only entries in
a request is made over HTTPS and a matching `.netrc` entry can be found, Go
adds HTTP Basic authentication credentials to the request. Go does not
authenticate requests made over HTTP. Go rejects HTTP-only entries in
`GOPROXY` that have embedded credentials.
In a future version, Go may add support for arbitrary authentication headers.
......
......@@ -20,7 +20,7 @@ the two is best for the job.
This page aims to define and organize our Go guidelines, based on our various
experiences. Several projects were started with different standards and they
can still have specifics. They will be described in their respective
can still have specifics. They are described in their respective
`README.md` or `PROCESS.md` files.
## Dependency Management
......@@ -89,7 +89,7 @@ projects:
## Code style and format
- Avoid global variables, even in packages. By doing so you will introduce side
- Avoid global variables, even in packages. By doing so you introduce side
effects if the package is included multiple times.
- Use `goimports` before committing.
[goimports](https://godoc.org/golang.org/x/tools/cmd/goimports)
......@@ -97,7 +97,7 @@ projects:
[Gofmt](https://golang.org/cmd/gofmt/), in addition to formatting import lines,
adding missing ones and removing unreferenced ones.
Most editors/IDEs will allow you to run commands before/after saving a file, you can set it
Most editors/IDEs allow you to run commands before/after saving a file, you can set it
up to run `goimports` so that it's applied to every file when saving.
- Place private methods below the first caller method in the source file.
......@@ -128,7 +128,7 @@ configuration of `golangci-lint`. All options for `golangci-lint` are listed in
this [example](https://github.com/golangci/golangci-lint/blob/master/.golangci.example.yml).
Once [recursive includes](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/56836)
become available, you will be able to share job templates like this
become available, you can share job templates like this
[analyzer](https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/analyzer.yml).
## Dependencies
......@@ -150,7 +150,7 @@ define and lock dependencies for reproducible builds. It should be used
whenever possible.
When Go Modules are in use, there should not be a `vendor/` directory. Instead,
Go will automatically download dependencies when they are needed to build the
Go automatically downloads dependencies when they are needed to build the
project. This is in line with how dependencies are handled with Bundler in Ruby
projects, and makes merge requests easier to review.
......@@ -177,7 +177,7 @@ databases.
In the rare event of managing a hosted database, it's necessary to use a
migration system like ActiveRecord is providing. A simple library like
[Journey](https://github.com/db-journey/journey), designed to be used in
`postgres` containers, can be deployed as long-running pods. New versions will
`postgres` containers, can be deployed as long-running pods. New versions
deploy a new pod, migrating the data automatically.
## Testing
......@@ -255,7 +255,7 @@ to make the test output easily readable.
to use for naming subtests. In the Go standard library, this is commonly the
`name string` field.
- Use `want`/`expect`/`actual` when you are specifying something in the
test case that will be used for assertion.
test case that is used for assertion.
#### Variable names
......@@ -414,7 +414,7 @@ builds](https://docs.docker.com/develop/develop-images/multistage-build/):
Generated Docker images should have the program at their `Entrypoint` to create
portable commands. That way, anyone can run the image, and without parameters
it will display its help message (if `cli` has been used).
it displays its help message (if `cli` has been used).
## Distributing Go binaries
......@@ -476,7 +476,7 @@ Example:
In case we want to drop support for `go 1.11` in GitLab `12.10`, we need to verify which Go versions we are using in `12.9`, `12.8`, and `12.7`.
We will not consider the active milestone, `12.10`, because a backport for `12.7` will be required in case of a critical security release.
We do not consider the active milestone, `12.10`, because a backport for `12.7` is required in case of a critical security release.
1. If both [Omnibus and CNG](#updating-go-version) were using Go `1.12` in GitLab `12.7` and later, then we safely drop support for `1.11`.
1. If Omnibus or CNG were using `1.11` in GitLab `12.7`, then we still need to keep support for Go `1.11` for easier backporting of security fixes.
......@@ -492,11 +492,11 @@ Use `goimports -local gitlab.com/gitlab-org` before committing.
is a tool that automatically formats Go source code using
[Gofmt](https://golang.org/cmd/gofmt/), in addition to formatting import lines,
adding missing ones and removing unreferenced ones.
By using the `-local gitlab.com/gitlab-org` option, `goimports` will group locally referenced
By using the `-local gitlab.com/gitlab-org` option, `goimports` groups locally referenced
packages separately from external ones. See
[the imports section](https://github.com/golang/go/wiki/CodeReviewComments#imports)
of the Code Review Comments page on the Go wiki for more details.
Most editors/IDEs will allow you to run commands before/after saving a file, you can set it
Most editors/IDEs allow you to run commands before/after saving a file, you can set it
up to run `goimports -local gitlab.com/gitlab-org` so that it's applied to every file when saving.
---
......
......@@ -30,7 +30,7 @@ project.feature_available?(:feature_symbol)
However, for features such as [Geo](../administration/geo/index.md) and
[Load balancing](../administration/database_load_balancing.md), which cannot be restricted
to only a subset of projects or namespaces, the check will be made directly in
to only a subset of projects or namespaces, the check is made directly in
the instance license.
1. Add the feature symbol on `EES_FEATURES`, `EEP_FEATURES` or `EEU_FEATURES` constants in
......
......@@ -6,15 +6,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Beginner's guide to writing end-to-end tests
In this tutorial, you will learn about the creation of end-to-end (_e2e_) tests
This tutorial walks you through the creation of end-to-end (_e2e_) tests
for [GitLab Community Edition](https://about.gitlab.com/install/?version=ce) and
[GitLab Enterprise Edition](https://about.gitlab.com/install/).
By the end of this tutorial, you will be able to:
By the end of this tutorial, you can:
- Determine whether an end-to-end test is needed.
- Understand the directory structure within `qa/`.
- Write a basic end-to-end test that will validate login features.
- Write a basic end-to-end test that validates login features.
- Develop any missing [page object](page_objects.md) libraries.
## Before you write a test
......@@ -68,17 +68,17 @@ The GitLab QA end-to-end tests are organized by the different
[stages in the DevOps lifecycle](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features/browser_ui).
Determine where the test should be placed by
[stage](https://about.gitlab.com/handbook/product/product-categories/#devops-stages),
determine which feature the test will belong to, and then place it in a subdirectory
determine which feature the test belongs to, and then place it in a subdirectory
under the stage.
![DevOps lifecycle by stages](img/gl-devops-lifecycle-by-stage-numbers_V12_10.png)
If the test is Enterprise Edition only, the test will be created in the `features/ee`
If the test is Enterprise Edition only, the test is created in the `features/ee`
directory, but follow the same DevOps lifecycle format.
## Create a skeleton test
In the first part of this tutorial we will be testing login, which is owned by the
In the first part of this tutorial we are testing login, which is owned by the
Manage stage. Inside `qa/specs/features/browser_ui/1_manage/login`, create a
file `basic_login_spec.rb`.
......@@ -286,12 +286,12 @@ end
Note the following important points:
- At the start of our example, we will be at the `page/issue/show.rb` [page](page_objects.md).
- At the start of our example, we are at the `page/issue/show.rb` [page](page_objects.md).
- Our test fabricates only what it needs, when it needs it.
- The issue is fabricated through the API to save time.
- GitLab prefers `let()` over instance variables. See
[best practices](../best_practices.md#subject-and-let-variables).
- `be_closed` is not implemented in `page/project/issue/show.rb` yet, but will be
- `be_closed` is not implemented in `page/project/issue/show.rb` yet, but is
implemented in the next step.
The issue is fabricated as a [Resource](resources.md), which is a GitLab entity
......
......@@ -23,7 +23,7 @@ We interpret user actions on the page to have some sort of effect. These actions
### Navigation
When a page is navigated to, there are elements that will always appear on the page unconditionally.
When a page is navigated to, there are elements that always appear on the page unconditionally.
Dynamic element validation is instituted when using
......@@ -100,7 +100,7 @@ Runtime::Browser.visit(:gitlab, Page::MyPage)
execute_stuff
```
will invoke GitLab QA to scan `MyPage` for `my_element` and `another_element` to be on the page before continuing to
invokes GitLab QA to scan `MyPage` for `my_element` and `another_element` to be on the page before continuing to
`execute_stuff`
### Clicking
......@@ -113,7 +113,7 @@ def open_layer
end
```
will invoke GitLab QA to ensure that `message_content` appears on
invokes GitLab QA to ensure that `message_content` appears on
the Layer upon clicking `my_element`.
This will imply that the Layer is indeed rendered before we continue our test.
This implies that the Layer is indeed rendered before we continue our test.
......@@ -65,4 +65,4 @@ If you want to run an `only: { :pipeline }` tagged test on your local GDK make s
Similarly to specifying that a test should only run against a specific environment, it's also possible to quarantine a
test only when it runs against a specific environment. The syntax is exactly the same, except that the `only: { ... }`
hash is nested in the [`quarantine: { ... }`](https://about.gitlab.com/handbook/engineering/quality/guidelines/debugging-qa-test-failures/#quarantining-tests) hash.
For instance, `quarantine: { only: { subdomain: :staging } }` will only quarantine the test when run against staging.
For instance, `quarantine: { only: { subdomain: :staging } }` only quarantines the test when run against staging.
......@@ -10,7 +10,7 @@ To run a specific test with a feature flag enabled you can use the `QA::Runtime:
enable and disable feature flags ([via the API](../../../api/features.md)).
Note that administrator authorization is required to change feature flags. `QA::Runtime::Feature`
will automatically authenticate as an administrator as long as you provide an appropriate access
automatically authenticates as an administrator as long as you provide an appropriate access
token via `GITLAB_QA_ADMIN_ACCESS_TOKEN` (recommended), or provide `GITLAB_ADMIN_USERNAME`
and `GITLAB_ADMIN_PASSWORD`.
......@@ -60,7 +60,7 @@ feature_group = "a_feature_group"
Runtime::Feature.enable(:feature_flag_name, feature_group: feature_group)
```
If no scope is provided, the feature flag will be set instance-wide:
If no scope is provided, the feature flag is set instance-wide:
```ruby
# This will affect all users!
......
......@@ -126,8 +126,8 @@ For example, when we [dequarantine](https://about.gitlab.com/handbook/engineerin
a flaky test we first want to make sure that it's no longer flaky.
We can do that using the `ce:custom-parallel` and `ee:custom-parallel` jobs.
Both are manual jobs that you can configure using custom variables.
When you click the name (not the play icon) of one of the parallel jobs,
you'll be prompted to enter variables. You can use any of [the variables
When clicking the name (not the play icon) of one of the parallel jobs,
you are prompted to enter variables. You can use any of [the variables
that can be used with `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-gitlab-environment-variables)
as well as these:
......@@ -137,7 +137,7 @@ as well as these:
| `QA_TESTS` | The test(s) to run (no default, which means run all the tests in the scenario). Use file paths as you would when running tests via RSpec, e.g., `qa/specs/features/ee/browser_ui` would include all the `EE` UI tests. |
| `QA_RSPEC_TAGS` | The RSpec tags to add (no default) |
For now [manual jobs with custom variables will not use the same variable
For now [manual jobs with custom variables don't use the same variable
when retried](https://gitlab.com/gitlab-org/gitlab/-/issues/31367), so if you want to run the same test(s) multiple times,
specify the same variables in each `custom-parallel` job (up to as
many of the 10 available jobs that you want to run).
......
......@@ -30,13 +30,13 @@ need to rely on volatile helpers or invoke Capybara methods directly. Imagine
invoking `fill_in :user_login` in every `*_spec.rb` file / test example.
When someone later changes `t.text_field :login` in the view associated with
this page to `t.text_field :username` it will generate a different field
this page to `t.text_field :username` it generates a different field
identifier, what would effectively break all tests.
Because we are using `Page::Main::Login.perform(&:sign_in_using_credentials)`
everywhere, when we want to sign in to GitLab, the page object is the single
source of truth, and we will need to update `fill_in :user_login`
to `fill_in :user_username` only in a one place.
source of truth, and we must update `fill_in :user_login`
to `fill_in :user_username` only in one place.
## What problems did we have in the past?
......@@ -44,7 +44,7 @@ We do not run QA tests for every commit, because of performance reasons, and
the time it would take to build packages and test everything.
That is why when someone changes `t.text_field :login` to
`t.text_field :username` in the _new session_ view we won't know about this
`t.text_field :username` in the _new session_ view we don't know about this
change until our GitLab QA nightly pipeline fails, or until someone triggers
`package-and-qa` action in their merge request.
......@@ -56,14 +56,14 @@ problem by introducing coupling between GitLab CE / EE views and GitLab QA.
## How did we solve fragile tests problem?
Currently, when you add a new `Page::Base` derived class, you will also need to
Currently, when you add a new `Page::Base` derived class, you must also
define all selectors that your page objects depend on.
Whenever you push your code to CE / EE repository, `qa:selectors` sanity test
job is going to be run as a part of a CI pipeline.
job runs as a part of a CI pipeline.
This test is going to validate all page objects that we have implemented in
`qa/page` directory. When it fails, you will be notified about missing
This test validates all page objects that we have implemented in
`qa/page` directory. When it fails, it notifies you about missing
or invalid views/selectors definition.
## How to properly implement a page object?
......@@ -95,10 +95,10 @@ end
### Defining Elements
The `view` DSL method will correspond to the Rails view, partial, or Vue component that renders the elements.
The `view` DSL method corresponds to the Rails view, partial, or Vue component that renders the elements.
The `element` DSL method in turn declares an element for which a corresponding
`data-qa-selector=element_name_snaked` data attribute will need to be added to the view file.
`data-qa-selector=element_name_snaked` data attribute must be added to the view file.
You can also define a value (String or Regexp) to match to the actual view
code but **this is deprecated** in favor of the above method for two reasons:
......@@ -257,7 +257,7 @@ These modules must:
1. Include/prepend other modules and define their `view`/`elements` in a `base.class_eval` block to ensure they're
defined in the class that prepends the module.
These steps ensure the sanity selectors check will detect problems properly.
These steps ensure the sanity selectors check detect problems properly.
For example, `qa/qa/ee/page/merge_request/show.rb` adds EE-specific methods to `qa/qa/page/merge_request/show.rb` (with
`QA::Page::MergeRequest::Show.prepend_if_ee('QA::EE::Page::MergeRequest::Show')`) and following is how it's implemented
......
......@@ -94,7 +94,7 @@ needs a group to be created in.
To define a resource attribute, you can use the `attribute` method with a
block using the other resource class to fabricate the resource.
That will allow access to the other resource from your resource object's
That allows access to the other resource from your resource object's
methods. You would usually use it in `#fabricate!`, `#api_get_path`,
`#api_post_path`, `#api_post_body`.
......@@ -144,7 +144,7 @@ end
```
**Note that all the attributes are lazily constructed. This means if you want
a specific attribute to be fabricated first, you'll need to call the
a specific attribute to be fabricated first, you must call the
attribute method first even if you're not using it.**
#### Product data attributes
......@@ -185,7 +185,7 @@ end
```
**Note again that all the attributes are lazily constructed. This means if
you call `shirt.brand` after moving to the other page, it'll not properly
you call `shirt.brand` after moving to the other page, it doesn't properly
retrieve the data because we're no longer on the expected page.**
Consider this:
......@@ -201,7 +201,7 @@ shirt.project.visit!
shirt.brand # => FAIL!
```
The above example will fail because now we're on the project page, trying to
The above example fails because now we're on the project page, trying to
construct the brand data from the shirt page, however we moved to the project
page already. There are two ways to solve this, one is that we could try to
retrieve the brand before visiting the project again:
......@@ -219,8 +219,8 @@ shirt.project.visit!
shirt.brand # => OK!
```
The attribute will be stored in the instance therefore all the following calls
will be fine, using the data previously constructed. If we think that this
The attribute is stored in the instance, therefore all the following calls
are fine, using the data previously constructed. If we think that this
might be too brittle, we could eagerly construct the data right before
ending fabrication:
......@@ -249,12 +249,12 @@ module QA
end
```
The `populate` method will iterate through its arguments and call each
The `populate` method iterates through its arguments and call each
attribute respectively. Here `populate(:brand)` has the same effect as
just `brand`. Using the populate method makes the intention clearer.
With this, it will make sure we construct the data right after we create the
shirt. The drawback is that this will always construct the data when the
With this, it ensures we construct the data right after we create the
shirt. The drawback is that this always constructs the data when the
resource is fabricated even if we don't need to use the data.
Alternatively, we could just make sure we're on the right page before
......@@ -290,7 +290,7 @@ module QA
end
```
This will make sure it's on the shirt page before constructing brand, and
This ensures it's on the shirt page before constructing brand, and
move back to the previous page to avoid breaking the state.
#### Define an attribute based on an API response
......@@ -343,16 +343,16 @@ end
- resource instance variables have the highest precedence
- attributes from the API response take precedence over attributes from the
block (usually from Browser UI)
- attributes without a value will raise a `QA::Resource::Base::NoValueError` error
- attributes without a value raises a `QA::Resource::Base::NoValueError` error
## Creating resources in your tests
To create a resource in your tests, you can call the `.fabricate!` method on
the resource class.
Note that if the resource class supports API fabrication, this will use this
Note that if the resource class supports API fabrication, this uses this
fabrication by default.
Here is an example that will use the API fabrication method under the hood
Here is an example that uses the API fabrication method under the hood
since it's supported by the `Shirt` resource class:
```ruby
......@@ -389,8 +389,7 @@ my_shirt = Resource::Shirt.fabricate_via_api! do |shirt|
end
```
In this case, the result will be similar to calling
`Resource::Shirt.fabricate!`.
In this case, the result is similar to calling `Resource::Shirt.fabricate!`.
## Where to ask for help?
......
......@@ -14,16 +14,16 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec
| Tag | Description |
|-----|-------------|
| `:elasticsearch` | The test requires an Elasticsearch service. It is used by the [instance-level scenario](https://gitlab.com/gitlab-org/gitlab-qa#definitions) [`Test::Integration::Elasticsearch`](https://gitlab.com/gitlab-org/gitlab/-/blob/72b62b51bdf513e2936301cb6c7c91ec27c35b4d/qa/qa/ee/scenario/test/integration/elasticsearch.rb) to include only tests that require Elasticsearch. |
| `:gitaly_cluster` | The test will run against a GitLab instance where repositories are stored on redundant Gitaly nodes behind a Praefect node. All nodes are [separate containers](../../../administration/gitaly/praefect.md#requirements-for-configuring-a-gitaly-cluster). Tests that use this tag have a longer setup time since there are three additional containers that need to be started. |
| `:jira` | The test requires a Jira Server. [GitLab-QA](https://gitlab.com/gitlab-org/gitlab-qa) will provision the Jira Server in a Docker container when the `Test::Integration::Jira` test scenario is run.
| `:kubernetes` | The test includes a GitLab instance that is configured to be run behind an SSH tunnel, allowing a TLS-accessible GitLab. This test will also include provisioning of at least one Kubernetes cluster to test against. _This tag is often be paired with `:orchestrated`._ |
| `:gitaly_cluster` | The test runs against a GitLab instance where repositories are stored on redundant Gitaly nodes behind a Praefect node. All nodes are [separate containers](../../../administration/gitaly/praefect.md#requirements-for-configuring-a-gitaly-cluster). Tests that use this tag have a longer setup time since there are three additional containers that need to be started. |
| `:jira` | The test requires a Jira Server. [GitLab-QA](https://gitlab.com/gitlab-org/gitlab-qa) provisions the Jira Server in a Docker container when the `Test::Integration::Jira` test scenario is run.
| `:kubernetes` | The test includes a GitLab instance that is configured to be run behind an SSH tunnel, allowing a TLS-accessible GitLab. This test also includes provisioning of at least one Kubernetes cluster to test against. _This tag is often be paired with `:orchestrated`._ |
| `:only` | The test is only to be run against specific environments or pipelines. See [Environment selection](environment_selection.md) for more information. |
| `:orchestrated` | The GitLab instance under test may be [configured by `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#orchestrated-tests) to be different to the default GitLab configuration, or `gitlab-qa` may launch additional services in separate Docker containers, or both. Tests tagged with `:orchestrated` are excluded when testing environments where we can't dynamically modify GitLab's configuration (for example, Staging). |
| `:quarantine` | The test has been [quarantined](https://about.gitlab.com/handbook/engineering/quality/guidelines/debugging-qa-test-failures/#quarantining-tests), will run in a separate job that only includes quarantined tests, and is allowed to fail. The test will be skipped in its regular job so that if it fails it will not hold up the pipeline. Note that you can also [quarantine a test only when it runs against specific environment](environment_selection.md#quarantining-a-test-for-a-specific-environment). |
| `:quarantine` | The test has been [quarantined](https://about.gitlab.com/handbook/engineering/quality/guidelines/debugging-qa-test-failures/#quarantining-tests), runs in a separate job that only includes quarantined tests, and is allowed to fail. The test is skipped in its regular job so that if it fails it doesn't hold up the pipeline. Note that you can also [quarantine a test only when it runs against specific environment](environment_selection.md#quarantining-a-test-for-a-specific-environment). |
| `:reliable` | The test has been [promoted to a reliable test](https://about.gitlab.com/handbook/engineering/quality/guidelines/reliable-tests/#promoting-an-existing-test-to-reliable) meaning it passes consistently in all pipelines, including merge requests. |
| `:requires_admin` | The test requires an admin account. Tests with the tag are excluded when run against Canary and Production environments. |
| `:runner` | The test depends on and will set up a GitLab Runner instance, typically to run a pipeline. |
| `:skip_live_env` | The test will be excluded when run against live deployed environments such as Staging, Canary, and Production. |
| `:runner` | The test depends on and sets up a GitLab Runner instance, typically to run a pipeline. |
| `:skip_live_env` | The test is excluded when run against live deployed environments such as Staging, Canary, and Production. |
| `:testcase` | The link to the test case issue in the [Quality Testcases project](https://gitlab.com/gitlab-org/quality/testcases/). |
| `:mattermost` | The test requires a GitLab Mattermost service on the GitLab instance. |
| `:ldap_no_server` | The test requires a GitLab instance to be configured to use LDAP. To be used with the `:orchestrated` tag. It does not spin up an LDAP server at orchestration time. Instead, it creates the LDAP server at runtime. |
......@@ -33,7 +33,7 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec
| `:smtp` | The test requires a GitLab instance to be configured to use an SMTP server. Tests SMTP notification email delivery from GitLab by using MailHog. |
| `:group_saml` | The test requires a GitLab instance that has SAML SSO enabled at the group level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. |
| `:instance_saml` | The test requires a GitLab instance that has SAML SSO enabled at the instance level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. |
| `:skip_signup_disabled` | The test uses UI to sign up a new user and will be skipped in any environment that does not allow new user registration via the UI. |
| `:skip_signup_disabled` | The test uses UI to sign up a new user and is skipped in any environment that does not allow new user registration via the UI. |
| `:smoke` | The test belongs to the test suite which verifies basic functionality of a GitLab instance.|
| `:github` | The test requires a GitHub personal access token. |
| `:repository_storage` | The test requires a GitLab instance to be configured to use multiple [repository storage paths](../../../administration/repository_storage_paths.md). Paired with the `:orchestrated` tag. |
......
......@@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
The [`jenkins_build_status_spec`](https://gitlab.com/gitlab-org/gitlab/blob/163c8a8c814db26d11e104d1cb2dcf02eb567dbe/qa/qa/specs/features/ee/browser_ui/3_create/jenkins/jenkins_build_status_spec.rb) spins up a Jenkins instance in a Docker container based on an image stored in the [GitLab-QA container registry](https://gitlab.com/gitlab-org/gitlab-qa/container_registry).
The Docker image it uses is preconfigured with some base data and plugins.
The test then configures the GitLab plugin in Jenkins with a URL of the GitLab instance that will be used
The test then configures the GitLab plugin in Jenkins with a URL of the GitLab instance that are used
to run the tests. Unfortunately, the GitLab Jenkins plugin does not accept ports so `http://localhost:3000` would
not be accepted. Therefore, this requires us to run GitLab on port 80 or inside a Docker container.
......@@ -30,7 +30,7 @@ To run the tests from the `/qa` directory:
CHROME_HEADLESS=false bin/qa Test::Instance::All http://localhost -- qa/specs/features/ee/browser_ui/3_create/jenkins/jenkins_build_status_spec.rb
```
The test will automatically spin up a Docker container for Jenkins and tear down once the test completes.
The test automatically spins up a Docker container for Jenkins and tear down once the test completes.
However, if you need to run Jenkins manually outside of the tests, use this command:
......@@ -43,7 +43,7 @@ docker run \
registry.gitlab.com/gitlab-org/gitlab-qa/jenkins-gitlab:version1
```
Jenkins will be available on `http://localhost:8080`.
Jenkins is available on `http://localhost:8080`.
Admin username is `admin` and password is `password`.
......@@ -59,19 +59,19 @@ the Docker Engine.
The tests tagged `:gitaly_ha` are orchestrated tests that can only be run against a set of Docker containers as configured and started by [the `Test::Integration::GitalyCluster` GitLab QA scenario](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#testintegrationgitalycluster-ceeefull-image-address).
As described in the documentation about the scenario noted above, the following command will run the tests:
As described in the documentation about the scenario noted above, the following command runs the tests:
```shell
gitlab-qa Test::Integration::GitalyCluster EE
```
However, that will remove the containers after it finishes running the tests. If you would like to do further testing, for example, if you would like to run a single test via a debugger, you can use [the `--no-tests` option](https://gitlab.com/gitlab-org/gitlab-qa#command-line-options) to make `gitlab-qa` skip running the tests, and to leave the containers running so that you can continue to use them.
However, that removes the containers after it finishes running the tests. If you would like to do further testing, for example, if you would like to run a single test via a debugger, you can use [the `--no-tests` option](https://gitlab.com/gitlab-org/gitlab-qa#command-line-options) to make `gitlab-qa` skip running the tests, and to leave the containers running so that you can continue to use them.
```shell
gitlab-qa Test::Integration::GitalyCluster EE --no-tests
```
When all the containers are running you will see the output of the `docker ps` command, showing on which ports the GitLab container can be accessed. For example:
When all the containers are running, the output of the `docker ps` command shows which ports the GitLab container can be accessed on. For example:
```plaintext
CONTAINER ID ... PORTS NAMES
......@@ -82,7 +82,7 @@ That shows that the GitLab instance running in the `gitlab-gitaly-ha` container
Another option is to use NGINX.
In both cases you will need to configure your machine to translate `gitlab-gitlab-ha.test` into an appropriate IP address:
In both cases you must configure your machine to translate `gitlab-gitlab-ha.test` into an appropriate IP address:
```shell
echo '127.0.0.1 gitlab-gitaly-ha.test' | sudo tee -a /etc/hosts
......@@ -205,7 +205,7 @@ You can now search through the logs for *Job log*, which matches delimited secti
A Job log is a subsection within these logs, related to app deployment. We use two jobs: `build` and `production`.
You can find the root causes of deployment failures in these logs, which can compromise the entire test.
If a `build` job fails, the `production` job won't run, and the test fails.
If a `build` job fails, the `production` job doesn't run, and the test fails.
The long test setup does not take screenshots of failures, which is a known [issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/270).
However, if the spec fails (after a successful deployment) then you should be able to find screenshots which display the feature failure.
......@@ -286,7 +286,7 @@ CHROME_HEADLESS=false bundle exec bin/qa QA::EE::Scenario::Test::Geo --primary-a
You can use [GitLab-QA Orchestrator](https://gitlab.com/gitlab-org/gitlab-qa) to orchestrate two GitLab containers and configure them as a Geo setup.
Geo requires an EE license. To visit the Geo sites in your browser, you will need a reverse proxy server (for example, [NGINX](https://www.nginx.com/)).
Geo requires an EE license. To visit the Geo sites in your browser, you need a reverse proxy server (for example, [NGINX](https://www.nginx.com/)).
1. Export your EE license
......@@ -296,7 +296,7 @@ Geo requires an EE license. To visit the Geo sites in your browser, you will nee
1. (Optional) Pull the GitLab image
This step is optional because pulling the Docker image is part of the [`Test::Integration::Geo` orchestrated scenario](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/d8c5c40607c2be0eda58bbca1b9f534b00889a0b/lib/gitlab/qa/scenario/test/integration/geo.rb). However, it's easier to monitor the download progress if you pull the image first, and the scenario will skip this step after checking that the image is up to date.
This step is optional because pulling the Docker image is part of the [`Test::Integration::Geo` orchestrated scenario](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/d8c5c40607c2be0eda58bbca1b9f534b00889a0b/lib/gitlab/qa/scenario/test/integration/geo.rb). However, it's easier to monitor the download progress if you pull the image first, and the scenario skips this step after checking that the image is up to date.
```shell
# For the most recent nightly image
......@@ -309,7 +309,7 @@ Geo requires an EE license. To visit the Geo sites in your browser, you will nee
docker pull registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:examplesha123456789
```
1. Run the [`Test::Integration::Geo` orchestrated scenario](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/d8c5c40607c2be0eda58bbca1b9f534b00889a0b/lib/gitlab/qa/scenario/test/integration/geo.rb) with the `--no-teardown` option to build the GitLab containers, configure the Geo setup, and run Geo end-to-end tests. Running the tests after the Geo setup is complete is optional; the containers will keep running after you stop the tests.
1. Run the [`Test::Integration::Geo` orchestrated scenario](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/d8c5c40607c2be0eda58bbca1b9f534b00889a0b/lib/gitlab/qa/scenario/test/integration/geo.rb) with the `--no-teardown` option to build the GitLab containers, configure the Geo setup, and run Geo end-to-end tests. Running the tests after the Geo setup is complete is optional; the containers keep running after you stop the tests.
```shell
# Using the most recent nightly image
......
......@@ -26,14 +26,14 @@ it 'should succeed', quarantine: 'https://gitlab.com/gitlab-org/gitlab/-/issues/
end
```
This means it will be skipped unless run with `--tag quarantine`:
This means it is skipped unless run with `--tag quarantine`:
```shell
bin/rspec --tag quarantine
```
**Before putting a test in quarantine, you should make sure that a
~"master:broken" issue exists for it so it won't stay in quarantine forever.**
~"master:broken" issue exists for it so it doesn't stay in quarantine forever.**
Once a test is in quarantine, there are 3 choices:
......
......@@ -11,7 +11,7 @@ in lieu of the standard Spec helper. Instead of `require 'spec_helper'`, use
`require 'rake_helper'`. The helper includes `spec_helper` for you, and configures
a few other things to make testing Rake tasks easier.
At a minimum, requiring the Rake helper will redirect `stdout`, include the
At a minimum, requiring the Rake helper redirects `stdout`, include the
runtime task helpers, and include the `RakeHelpers` Spec support module.
The `RakeHelpers` module exposes a `run_rake_task(<task>)` method to make
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment