Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
40bbc9d9
Commit
40bbc9d9
authored
Aug 08, 2017
by
Rémy Coutable
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Document rspec-retry and rspec-flaky
Signed-off-by:
Rémy Coutable
<
remy@rymai.me
>
parent
57a561db
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
20 additions
and
3 deletions
+20
-3
doc/development/testing.md
doc/development/testing.md
+20
-3
No files found.
doc/development/testing.md
View file @
40bbc9d9
...
...
@@ -157,8 +157,9 @@ trade-off:
-
Unit tests are usually cheap, and you should consider them like the basement
of your house: you need them to be confident that your code is behaving
correctly. However if you run only unit tests without integration / system tests, you might [miss] the [big] [picture]!
-
Integration tests are a bit more expensive, but don't abuse them. A feature test
correctly. However if you run only unit tests without integration / system
tests, you might [miss] the [big] [picture]!
-
Integration tests are a bit more expensive, but don't abuse them. A system test
is often better than an integration test that is stubbing a lot of internals.
-
System tests are expensive (compared to unit tests), even more if they require
a JavaScript driver. Make sure to follow the guidelines in the
[
Speed
](
#test-speed
)
...
...
@@ -195,11 +196,27 @@ Please consult the [dedicated "Frontend testing" guide](./fe_guide/testing.md).
-
Try to match the ordering of tests to the ordering within the class.
-
Try to follow the
[
Four-Phase Test
][
four-phase-test
]
pattern, using newlines
to separate phases.
-
Try to use
`Gitlab.config.gitlab.host`
rather than hard coding
`'localhost'`
-
Use
`Gitlab.config.gitlab.host`
rather than hard coding
`'localhost'`
-
Don't assert against the absolute value of a sequence-generated attribute (see
[
Gotchas
](
gotchas.md#dont-assert-against-the-absolute-value-of-a-sequence-generated-attribute
)
).
-
Don't supply the
`:each`
argument to hooks since it's the default.
-
On
`before`
and
`after`
hooks, prefer it scoped to
`:context`
over
`:all`
[
four-phase-test
]:
https://robots.thoughtbot.com/four-phase-test
### Automatic retries and flaky tests detection
On our CI, we use [rspec-retry] to automatically retry a failing example a few
times (see [
`spec/spec_helper.rb`
] for the precise retries count).
We also use a home-made
`RspecFlaky::Listener`
listener which records flaky
examples in a JSON report file on
`master`
(
`retrieve-tests-metadata`
and
`update-tests-metadata`
jobs), and warns when a new flaky example
is detected in any other branch (
`flaky-examples-check`
job). In the future, the
`flaky-examples-check`
job will not be allowed to fail.
[
rspec-retry
]:
https://github.com/NoRedInk/rspec-retry
[
`spec/spec_helper.rb`
]:
https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/spec_helper.rb
### `let` variables
GitLab's RSpec suite has made extensive use of
`let`
variables to reduce
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment