Commit 7f3301c3 authored by Marcel Amirault's avatar Marcel Amirault

Merge branch 'eread/refactor-more-rake-task-topics' into 'master'

Further refactoring of Rake task pages

Closes #215107

See merge request gitlab-org/gitlab!30010
parents 5a1e492b 50781cb4
...@@ -14,5 +14,7 @@ swap: ...@@ -14,5 +14,7 @@ swap:
postgres: PostgreSQL postgres: PostgreSQL
raketask: Rake task raketask: Rake task
raketasks: Rake tasks raketasks: Rake tasks
rspec: RSpec
self hosted: self-managed self hosted: self-managed
self-hosted: self-managed self-hosted: self-managed
yaml: YAML
...@@ -240,10 +240,10 @@ to those documents for details. ...@@ -240,10 +240,10 @@ to those documents for details.
helm init --service-account tiller helm init --service-account tiller
``` ```
- Copy the file <https://gitlab.com/gitlab-org/charts/gitlab/raw/master/examples/values-minikube-minimum.yaml> - Copy the [Minikube minimum values YAML file](https://gitlab.com/gitlab-org/charts/gitlab/raw/master/examples/values-minikube-minimum.yaml)
to your workstation. to your workstation.
- Find the IP address in the output of `minikube ip` and update the yaml file with - Find the IP address in the output of `minikube ip` and update the YAML file with
this IP address. this IP address.
- Install the GitLab Helm Chart: - Install the GitLab Helm Chart:
...@@ -254,7 +254,7 @@ to those documents for details. ...@@ -254,7 +254,7 @@ to those documents for details.
``` ```
If you want to modify some GitLab settings, you can use the above-mentioned config If you want to modify some GitLab settings, you can use the above-mentioned config
as a base and create your own yaml file. as a base and create your own YAML file.
- Monitor the installation progress via `helm status gitlab` and `minikube dashboard`. - Monitor the installation progress via `helm status gitlab` and `minikube dashboard`.
The installation could take up to 20-30 minutes depending on the amount of resources The installation could take up to 20-30 minutes depending on the amount of resources
...@@ -263,7 +263,7 @@ to those documents for details. ...@@ -263,7 +263,7 @@ to those documents for details.
- When all the pods show either a `Running` or `Completed` status, get the GitLab password as - When all the pods show either a `Running` or `Completed` status, get the GitLab password as
described in [Initial login](https://docs.gitlab.com/charts/installation/deployment.html#initial-login), described in [Initial login](https://docs.gitlab.com/charts/installation/deployment.html#initial-login),
and log in to GitLab via the UI. It will be accessible via `https://gitlab.domain` and log in to GitLab via the UI. It will be accessible via `https://gitlab.domain`
where `domain` is the value provided in the yaml file. where `domain` is the value provided in the YAML file.
<!-- ## Troubleshooting <!-- ## Troubleshooting
......
...@@ -5,27 +5,33 @@ run into some head-banging database problems. ...@@ -5,27 +5,33 @@ run into some head-banging database problems.
An easy first step is to search for your error in Slack, or search for `GitLab <my error>` with Google. An easy first step is to search for your error in Slack, or search for `GitLab <my error>` with Google.
--- Available `RAILS_ENV`:
Available `RAILS_ENV` - `production` (generally not for your main GDK database, but you may need this for other installations such as Omnibus).
- `development` (this is your main GDK db).
- `production` (generally not for your main GDK db, but you may need this for e.g. Omnibus) - `test` (used for tests like RSpec).
- `development` (this is your main GDK db)
- `test` (used for tests like rspec)
## Nuke everything and start over ## Nuke everything and start over
If you just want to delete everything and start over with an empty DB (~1 minute): If you just want to delete everything and start over with an empty DB (approximately 1 minute):
- `bundle exec rake db:reset RAILS_ENV=development` ```shell
bundle exec rake db:reset RAILS_ENV=development
```
If you just want to delete everything and start over with dummy data (~4 minutes). This also does `db:reset` and runs DB-specific migrations: If you just want to delete everything and start over with dummy data (approximately 4 minutes). This
also does `db:reset` and runs DB-specific migrations:
- `bundle exec rake dev:setup RAILS_ENV=development` ```shell
bundle exec rake dev:setup RAILS_ENV=development
```
If your test DB is giving you problems, it is safe to nuke it because it doesn't contain important data: If your test DB is giving you problems, it is safe to nuke it because it doesn't contain important
data:
- `bundle exec rake db:reset RAILS_ENV=test` ```shell
bundle exec rake db:reset RAILS_ENV=test
```
## Migration wrangling ## Migration wrangling
......
...@@ -277,7 +277,7 @@ You can combine one or more of the following: ...@@ -277,7 +277,7 @@ You can combine one or more of the following:
### GitLab `/help` tests ### GitLab `/help` tests
Several [rspec tests](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/features/help_pages_spec.rb) Several [RSpec tests](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/features/help_pages_spec.rb)
are run to ensure GitLab documentation renders and works correctly. In particular, that [main docs landing page](../../README.md) will work correctly from `/help`. are run to ensure GitLab documentation renders and works correctly. In particular, that [main docs landing page](../../README.md) will work correctly from `/help`.
For example, [GitLab.com's `/help`](https://gitlab.com/help). For example, [GitLab.com's `/help`](https://gitlab.com/help).
......
...@@ -77,7 +77,7 @@ If your test-suite is failing with Gitaly issues, as a first step, try running: ...@@ -77,7 +77,7 @@ If your test-suite is failing with Gitaly issues, as a first step, try running:
rm -rf tmp/tests/gitaly rm -rf tmp/tests/gitaly
``` ```
During rspec tests, the Gitaly instance will write logs to `gitlab/log/gitaly-test.log`. During RSpec tests, the Gitaly instance will write logs to `gitlab/log/gitaly-test.log`.
## Legacy Rugged code ## Legacy Rugged code
......
# Mass Inserting Rails Models # Mass inserting Rails models
Setting the environment variable [`MASS_INSERT=1`](rake_tasks.md#env-variables) Setting the environment variable [`MASS_INSERT=1`](rake_tasks.md#environment-variables)
when running `rake setup` will create millions of records, but these records when running [`rake setup`](rake_tasks.md) will create millions of records, but these records
aren't visible to the `root` user by default. aren't visible to the `root` user by default.
To make any number of the mass-inserted projects visible to the `root` user, run To make any number of the mass-inserted projects visible to the `root` user, run
......
...@@ -87,7 +87,7 @@ printer.print(File.open('/tmp/profile.html', 'w')) ...@@ -87,7 +87,7 @@ printer.print(File.open('/tmp/profile.html', 'w'))
[GitLab-Profiler](https://gitlab.com/gitlab-com/gitlab-profiler) is a project [GitLab-Profiler](https://gitlab.com/gitlab-com/gitlab-profiler) is a project
that builds on this to add some additional niceties, such as allowing that builds on this to add some additional niceties, such as allowing
configuration with a single Yaml file for multiple URLs, and uploading of the configuration with a single YAML file for multiple URLs, and uploading of the
profile and log output to S3. profile and log output to S3.
For GitLab.com, you can find the latest results here (restricted to GitLab Team members only): For GitLab.com, you can find the latest results here (restricted to GitLab Team members only):
......
# Rake tasks for developers # Rake tasks for developers
## Set up db with developer seeds Rake tasks are available for developers and others contributing to GitLab.
Note that if your db user does not have advanced privileges you must create the db manually before running this command. ## Set up database with developer seeds
Note that if your database user does not have advanced privileges, you must create the database manually before running this command.
```shell ```shell
bundle exec rake setup bundle exec rake setup
...@@ -12,13 +14,15 @@ The `setup` task is an alias for `gitlab:setup`. ...@@ -12,13 +14,15 @@ The `setup` task is an alias for `gitlab:setup`.
This tasks calls `db:reset` to create the database, and calls `db:seed_fu` to seed the database. This tasks calls `db:reset` to create the database, and calls `db:seed_fu` to seed the database.
Note: `db:setup` calls `db:seed` but this does nothing. Note: `db:setup` calls `db:seed` but this does nothing.
### Env variables ### Environment variables
**MASS_INSERT**: Create millions of users (2m), projects (5m) and its **MASS_INSERT**: Create millions of users (2m), projects (5m) and its
relations. It's highly recommended to run the seed with it to catch slow queries relations. It's highly recommended to run the seed with it to catch slow queries
while developing. Expect the process to take up to 20 extra minutes. while developing. Expect the process to take up to 20 extra minutes.
**LARGE_PROJECTS**: Create large projects (through import) from a predefined set of urls. See also [Mass inserting Rails models](mass_insert.md).
**LARGE_PROJECTS**: Create large projects (through import) from a predefined set of URLs.
### Seeding issues for all or a given project ### Seeding issues for all or a given project
...@@ -86,7 +90,7 @@ echo 'yes' | bundle exec rake setup ...@@ -86,7 +90,7 @@ echo 'yes' | bundle exec rake setup
To save you from answering `yes` manually. To save you from answering `yes` manually.
### Discard stdout ### Discard `stdout`
Since the script would print a lot of information, it could be slowing down Since the script would print a lot of information, it could be slowing down
your terminal, and it would generate more than 20G logs if you just redirect your terminal, and it would generate more than 20G logs if you just redirect
...@@ -97,8 +101,8 @@ it to a file. If we don't care about the output, we could just redirect it to ...@@ -97,8 +101,8 @@ it to a file. If we don't care about the output, we could just redirect it to
echo 'yes' | bundle exec rake setup > /dev/null echo 'yes' | bundle exec rake setup > /dev/null
``` ```
Note that since you can't see the questions from stdout, you might just want Note that since you can't see the questions from `stdout`, you might just want
to `echo 'yes'` to keep it running. It would still print the errors on stderr to `echo 'yes'` to keep it running. It would still print the errors on `stderr`
so no worries about missing errors. so no worries about missing errors.
### Extra Project seed options ### Extra Project seed options
...@@ -113,33 +117,35 @@ There are a few environment flags you can pass to change how projects are seeded ...@@ -113,33 +117,35 @@ There are a few environment flags you can pass to change how projects are seeded
In order to run the test you can use the following commands: In order to run the test you can use the following commands:
- `bin/rake spec` to run the rspec suite - `bin/rake spec` to run the RSpec suite
- `bin/rake spec:unit` to run only the unit tests - `bin/rake spec:unit` to run only the unit tests
- `bin/rake spec:integration` to run only the integration tests - `bin/rake spec:integration` to run only the integration tests
- `bin/rake spec:system` to run only the system tests - `bin/rake spec:system` to run only the system tests
- `bin/rake karma` to run the Karma test suite - `bin/rake karma` to run the Karma test suite
Note: `bin/rake spec` takes significant time to pass. `bin/rake spec` takes significant time to pass.
Instead of running full test suite locally you can save a lot of time by running Instead of running the full test suite locally, you can save a lot of time by running
a single test or directory related to your changes. After you submit merge request a single test or directory related to your changes. After you submit a merge request,
CI will run full test suite for you. Green CI status in the merge request means CI will run full test suite for you. Green CI status in the merge request means
full test suite is passed. full test suite is passed.
Note: You can't run `rspec .` since this will try to run all the `_spec.rb` You can't run `rspec .` since this will try to run all the `_spec.rb`
files it can find, also the ones in `/tmp` files it can find, also the ones in `/tmp`
Note: You can pass RSpec command line options to the `spec:unit`, You can pass RSpec command line options to the `spec:unit`,
`spec:integration`, and `spec:system` tasks, e.g. `bin/rake "spec:unit[--tag ~geo --dry-run]"`. `spec:integration`, and `spec:system` tasks. For example, `bin/rake "spec:unit[--tag ~geo --dry-run]"`.
To run a single test file you can use: For an RSpec test, to run a single test file you can run:
- `bin/rspec spec/controllers/commit_controller_spec.rb` for a rspec test ```shell
bin/rspec spec/controllers/commit_controller_spec.rb
```
To run several tests inside one directory: To run several tests inside one directory:
- `bin/rspec spec/requests/api/` for the rspec tests if you want to test API only - `bin/rspec spec/requests/api/` for the RSpec tests if you want to test API only
### Speed-up tests, Rake tasks, and migrations ### Speed up tests, Rake tasks, and migrations
[Spring](https://github.com/rails/spring) is a Rails application preloader. It [Spring](https://github.com/rails/spring) is a Rails application preloader. It
speeds up development by keeping your application running in the background so speeds up development by keeping your application running in the background so
...@@ -172,18 +178,16 @@ This will compile and minify all JavaScript and CSS assets and copy them along ...@@ -172,18 +178,16 @@ This will compile and minify all JavaScript and CSS assets and copy them along
with all other frontend assets (images, fonts, etc) into `/public/assets` where with all other frontend assets (images, fonts, etc) into `/public/assets` where
they can be easily inspected. they can be easily inspected.
## Updating Emoji Aliases ## Emoji tasks
To update the Emoji aliases file (used for Emoji autocomplete) you must run the To update the Emoji aliases file (used for Emoji autocomplete), run the
following: following:
```shell ```shell
bundle exec rake gemojione:aliases bundle exec rake gemojione:aliases
``` ```
## Updating Emoji Digests To update the Emoji digests file (used for Emoji autocomplete), run the
To update the Emoji digests file (used for Emoji autocomplete) you must run the
following: following:
```shell ```shell
...@@ -193,9 +197,7 @@ bundle exec rake gemojione:digests ...@@ -193,9 +197,7 @@ bundle exec rake gemojione:digests
This will update the file `fixtures/emojis/digests.json` based on the currently This will update the file `fixtures/emojis/digests.json` based on the currently
available Emoji. available Emoji.
## Emoji Sprites To generate a sprite file containing all the Emoji, run:
Generating a sprite file containing all the Emoji can be done by running:
```shell ```shell
bundle exec rake gemojione:sprite bundle exec rake gemojione:sprite
...@@ -206,7 +208,7 @@ such changes, first generate the `emoji.png` spritesheet with the above Rake ...@@ -206,7 +208,7 @@ such changes, first generate the `emoji.png` spritesheet with the above Rake
task, then check the dimensions of the new spritesheet and update the task, then check the dimensions of the new spritesheet and update the
`SPRITESHEET_WIDTH` and `SPRITESHEET_HEIGHT` constants accordingly. `SPRITESHEET_WIDTH` and `SPRITESHEET_HEIGHT` constants accordingly.
## Updating project templates ## Update project templates
Starting a project from a template needs this project to be exported. On a Starting a project from a template needs this project to be exported. On a
up to date master branch run: up to date master branch run:
...@@ -249,7 +251,7 @@ bundle exec rake db:obsolete_ignored_columns ...@@ -249,7 +251,7 @@ bundle exec rake db:obsolete_ignored_columns
Feel free to remove their definitions from their `ignored_columns` definitions. Feel free to remove their definitions from their `ignored_columns` definitions.
## Update GraphQL Documentation and Schema definitions ## Update GraphQL documentation and schema definitions
To generate GraphQL documentation based on the GitLab schema, run: To generate GraphQL documentation based on the GitLab schema, run:
......
...@@ -32,7 +32,7 @@ Here are some things to keep in mind regarding test performance: ...@@ -32,7 +32,7 @@ Here are some things to keep in mind regarding test performance:
## RSpec ## RSpec
To run rspec tests: To run RSpec tests:
```shell ```shell
# run all tests # run all tests
...@@ -491,7 +491,7 @@ range of inputs. By specifying the test case once, alongside a table of inputs ...@@ -491,7 +491,7 @@ range of inputs. By specifying the test case once, alongside a table of inputs
and the expected output for each, your tests can be made easier to read and more and the expected output for each, your tests can be made easier to read and more
compact. compact.
We use the [rspec-parameterized](https://github.com/tomykaira/rspec-parameterized) We use the [RSpec::Parameterized](https://github.com/tomykaira/rspec-parameterized)
gem. A short example, using the table syntax and checking Ruby equality for a gem. A short example, using the table syntax and checking Ruby equality for a
range of inputs, might look like this: range of inputs, might look like this:
...@@ -524,7 +524,7 @@ objects, FactoryBot-created objects etc. can lead to ...@@ -524,7 +524,7 @@ objects, FactoryBot-created objects etc. can lead to
### Prometheus tests ### Prometheus tests
Prometheus metrics may be preserved from one test run to another. To ensure that metrics are Prometheus metrics may be preserved from one test run to another. To ensure that metrics are
reset before each example, add the `:prometheus` tag to the Rspec test. reset before each example, add the `:prometheus` tag to the RSpec test.
### Matchers ### Matchers
......
...@@ -40,7 +40,7 @@ Quarantined tests are run on the CI in dedicated jobs that are allowed to fail: ...@@ -40,7 +40,7 @@ Quarantined tests are run on the CI in dedicated jobs that are allowed to fail:
## Automatic retries and flaky tests detection ## Automatic retries and flaky tests detection
On our CI, we use [rspec-retry](https://github.com/NoRedInk/rspec-retry) to automatically retry a failing example a few On our CI, we use [RSpec::Retry](https://github.com/NoRedInk/rspec-retry) to automatically retry a failing example a few
times (see [`spec/spec_helper.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/spec_helper.rb) for the precise retries count). times (see [`spec/spec_helper.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/spec_helper.rb) for the precise retries count).
We also use a home-made `RspecFlaky::Listener` listener which records flaky We also use a home-made `RspecFlaky::Listener` listener which records flaky
......
...@@ -235,6 +235,8 @@ To index via the Admin Area: ...@@ -235,6 +235,8 @@ To index via the Admin Area:
### Indexing through Rake tasks ### Indexing through Rake tasks
Indexing can be performed using Rake tasks.
#### Indexing small instances #### Indexing small instances
CAUTION: **Warning**: CAUTION: **Warning**:
...@@ -400,40 +402,32 @@ or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq ...@@ -400,40 +402,32 @@ or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq
For repository and snippet files, GitLab will only index up to 1 MiB of content, in order to avoid indexing timeouts. For repository and snippet files, GitLab will only index up to 1 MiB of content, in order to avoid indexing timeouts.
## GitLab Elasticsearch Rake Tasks ## GitLab Elasticsearch Rake tasks
There are several Rake tasks available to you via the command line: Rake tasks are available to:
- [`sudo gitlab-rake gitlab:elastic:index`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) - [Build and install](#building-and-installing) the indexer.
- This is a wrapper task. It does the following: - Delete indexes when [disabling Elasticsearch](#disabling-elasticsearch).
- `sudo gitlab-rake gitlab:elastic:recreate_index` - [Add GitLab data](#adding-gitlabs-data-to-the-elasticsearch-index) to an index.
- `sudo gitlab-rake gitlab:elastic:clear_index_status`
- `sudo gitlab-rake gitlab:elastic:index_projects` The following are some available Rake tasks:
- `sudo gitlab-rake gitlab:elastic:index_snippets`
- [`sudo gitlab-rake gitlab:elastic:index_projects`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Task | Description |
- This iterates over all projects and queues Sidekiq jobs to index them in the background. |:--------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
- [`sudo gitlab-rake gitlab:elastic:index_projects_status`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | [`sudo gitlab-rake gitlab:elastic:index`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Wrapper task for `gitlab:elastic:create_empty_index`, `gitlab:elastic:clear_index_status`, `gitlab:elastic:index_projects`, and `gitlab:elastic:index_snippets`. |
- This determines the overall status of the indexing. It is done by counting the total number of indexed projects, dividing by a count of the total number of projects, then multiplying by 100. | [`sudo gitlab-rake gitlab:elastic:index_projects`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Iterates over all projects and queues Sidekiq jobs to index them in the background. |
- [`sudo gitlab-rake gitlab:elastic:clear_index_status`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | [`sudo gitlab-rake gitlab:elastic:index_projects_status`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Determines the overall status of the indexing. It is done by counting the total number of indexed projects, dividing by a count of the total number of projects, then multiplying by 100. |
- This deletes all instances of IndexStatus for all projects. | [`sudo gitlab-rake gitlab:elastic:clear_index_status`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Deletes all instances of IndexStatus for all projects. |
| [`sudo gitlab-rake gitlab:elastic:create_empty_index[<INDEX_NAME>]`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Generates an empty index on the Elasticsearch side only if it doesn't already exist. |
NOTE: **Note:** | [`sudo gitlab-rake gitlab:elastic:delete_index[<INDEX_NAME>]`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Removes the GitLab index on the Elasticsearch instance. |
The `INDEX_NAME` parameter is optional and will use the default index name from the current `RAILS_ENV` if not set. | [`sudo gitlab-rake gitlab:elastic:recreate_index[<INDEX_NAME>]`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Wrapper task for `gitlab:elastic:delete_index[<INDEX_NAME>]` and `gitlab:elastic:create_empty_index[<INDEX_NAME>]`. |
| [`sudo gitlab-rake gitlab:elastic:index_snippets`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Performs an Elasticsearch import that indexes the snippets data. |
- [`sudo gitlab-rake gitlab:elastic:create_empty_index[<INDEX_NAME>]`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | [`sudo gitlab-rake gitlab:elastic:projects_not_indexed`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Displays which projects are not indexed. |
- This generates an empty index on the Elasticsearch side only if it doesn't already exists.
- [`sudo gitlab-rake gitlab:elastic:delete_index[<INDEX_NAME>]`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) NOTE: **Note:**
- This removes the GitLab index on the Elasticsearch instance. The `INDEX_NAME` parameter is optional and will use the default index name from the current `RAILS_ENV` if not set.
- [`sudo gitlab-rake gitlab:elastic:recreate_index[<INDEX_NAME>]`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake)
- This is a wrapper task. It does the following: ### Environment variables
- `sudo gitlab-rake gitlab:elastic:delete_index[<INDEX_NAME>]`
- `sudo gitlab-rake gitlab:elastic:create_empty_index[<INDEX_NAME>]`
- [`sudo gitlab-rake gitlab:elastic:index_snippets`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake)
- Performs an Elasticsearch import that indexes the snippets data.
- [`sudo gitlab-rake gitlab:elastic:projects_not_indexed`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake)
- Displays which projects are not indexed.
### Environment Variables
In addition to the Rake tasks, there are some environment variables that can be used to modify the process: In addition to the Rake tasks, there are some environment variables that can be used to modify the process:
...@@ -453,7 +447,7 @@ Indexing project repositories...I, [2019-03-04T21:27:03.083410 #3384] INFO -- : ...@@ -453,7 +447,7 @@ Indexing project repositories...I, [2019-03-04T21:27:03.083410 #3384] INFO -- :
I, [2019-03-04T21:27:05.215266 #3384] INFO -- : Indexing GitLab User / test (ID=33) is done! I, [2019-03-04T21:27:05.215266 #3384] INFO -- : Indexing GitLab User / test (ID=33) is done!
``` ```
## Elasticsearch Index Scopes ## Elasticsearch index scopes
When performing a search, the GitLab index will use the following scopes: When performing a search, the GitLab index will use the following scopes:
......
...@@ -18,7 +18,7 @@ The following are available Rake tasks: ...@@ -18,7 +18,7 @@ The following are available Rake tasks:
| Tasks | Description | | Tasks | Description |
|:-------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------| |:-------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------|
| [Back up and restore](backup_restore.md) | Back up, restore, and migrate GitLab instances between servers. | | [Back up and restore](backup_restore.md) | Back up, restore, and migrate GitLab instances between servers. |
| [Clean up](cleanup.md) | Clean up unneeded items GitLab instances. | | [Clean up](cleanup.md) | Clean up unneeded items from GitLab instances. |
| [Development](../development/rake_tasks.md) | Tasks for GitLab contributors. | | [Development](../development/rake_tasks.md) | Tasks for GitLab contributors. |
| [Elasticsearch](../integration/elasticsearch.md#gitlab-elasticsearch-rake-tasks) | Maintain Elasticsearch in a GitLab instance. | | [Elasticsearch](../integration/elasticsearch.md#gitlab-elasticsearch-rake-tasks) | Maintain Elasticsearch in a GitLab instance. |
| [Enable namespaces](features.md) | Enable usernames and namespaces for user projects. | | [Enable namespaces](features.md) | Enable usernames and namespaces for user projects. |
...@@ -31,6 +31,7 @@ The following are available Rake tasks: ...@@ -31,6 +31,7 @@ The following are available Rake tasks:
| [LDAP maintenance](../administration/raketasks/ldap.md) | [LDAP](../administration/auth/ldap.md)-related tasks. | | [LDAP maintenance](../administration/raketasks/ldap.md) | [LDAP](../administration/auth/ldap.md)-related tasks. |
| [List repositories](list_repos.md) | List of all GitLab-managed Git repositories on disk. | | [List repositories](list_repos.md) | List of all GitLab-managed Git repositories on disk. |
| [Project import/export](../administration/raketasks/project_import_export.md) | Prepare for [project exports and imports](../user/project/settings/import_export.md). | | [Project import/export](../administration/raketasks/project_import_export.md) | Prepare for [project exports and imports](../user/project/settings/import_export.md). |
| [Sample Prometheus data](generate_sample_prometheus_data.md) | Generate sample Prometheus data. |
| [Repository storage](../administration/raketasks/storage.md) | List and migrate existing projects and attachments from legacy storage to hashed storage. | | [Repository storage](../administration/raketasks/storage.md) | List and migrate existing projects and attachments from legacy storage to hashed storage. |
| [Uploads migrate](../administration/raketasks/uploads/migrate.md) | Migrate uploads between storage local and object storage. | | [Uploads migrate](../administration/raketasks/uploads/migrate.md) | Migrate uploads between storage local and object storage. |
| [Uploads sanitize](../administration/raketasks/uploads/sanitize.md) | Remove EXIF data from images uploaded to earlier versions of GitLab. | | [Uploads sanitize](../administration/raketasks/uploads/sanitize.md) | Remove EXIF data from images uploaded to earlier versions of GitLab. |
......
# Namespaces # Namespaces
This Rake task enables [namespaces](../user/group/index.md#namespaces) for projects.
## Enable usernames and namespaces for user projects ## Enable usernames and namespaces for user projects
This command will enable the namespaces feature introduced in v4.0. It will move every project in its namespace folder. This command will enable the namespaces feature introduced in GitLab 4.0. It will move every project in its namespace folder.
Note: Note:
- Because the **repository location will change**, you will need to **update all your Git URLs** to point to the new location. - The **repository location will change**, so you will need to **update all your Git URLs** to
- Username can be changed at **Profile ➔ Account**. point to the new location.
- The username can be changed at **Profile > Account**.
**Example:**
Old path: `git@example.org:myrepo.git` For example:
New path: `git@example.org:username/myrepo.git` or `git@example.org:groupname/myrepo.git` - Old path: `git@example.org:myrepo.git`.
- New path: `git@example.org:username/myrepo.git` or `git@example.org:groupname/myrepo.git`.
```shell ```shell
bundle exec rake gitlab:enable_namespaces RAILS_ENV=production bundle exec rake gitlab:enable_namespaces RAILS_ENV=production
......
# Generate Sample Prometheus Data # Generate sample Prometheus data
This command will run Prometheus queries for each of the metrics of a specific environment This command will run Prometheus queries for each of the metrics of a specific environment
for a series of time intervals: 30 minutes, 3 hours, 8 hours, 24 hours, 72 hours, and 7 days for a series of time intervals to now:
to now. The results of each of query are stored under a `sample_metrics` directory as a yaml
- 30 minutes
- 3 hours
- 8 hours
- 24 hours
- 72 hours
- 7 days
The results of each of query are stored under a `sample_metrics` directory as a YAML
file named by the metric's `identifier`. When the environmental variable `USE_SAMPLE_METRICS` file named by the metric's `identifier`. When the environmental variable `USE_SAMPLE_METRICS`
is set, the Prometheus API query is re-routed to `Projects::Environments::SampleMetricsController` is set, the Prometheus API query is re-routed to `Projects::Environments::SampleMetricsController`
which loads the appropriate data set if it is present within the `sample_metrics` directory. which loads the appropriate data set if it is present within the `sample_metrics` directory.
- This command requires an id from an Environment with an available Prometheus installation. This command requires an ID from an environment with an available Prometheus installation.
## Example
**Example:** The following example demonstrates how to run the Rake task:
```shell ```shell
bundle exec rake gitlab:generate_sample_prometheus_data[21] bundle exec rake gitlab:generate_sample_prometheus_data[21]
......
...@@ -532,7 +532,7 @@ Then install it on your cluster: ...@@ -532,7 +532,7 @@ Then install it on your cluster:
helm install --name kibana stable/elastic-stack --values kibana.yml helm install --name kibana stable/elastic-stack --values kibana.yml
``` ```
To access kibana, forward the port to your local machine: To access Kibana, forward the port to your local machine:
```shell ```shell
kubectl port-forward svc/kibana 5601:443 kubectl port-forward svc/kibana 5601:443
...@@ -775,7 +775,7 @@ In order for GitLab Runner to function, you **must** specify the following: ...@@ -775,7 +775,7 @@ In order for GitLab Runner to function, you **must** specify the following:
- `runnerRegistrationToken` - The registration token for adding new Runners to GitLab. This must be - `runnerRegistrationToken` - The registration token for adding new Runners to GitLab. This must be
[retrieved from your GitLab instance](../../ci/runners/README.md). [retrieved from your GitLab instance](../../ci/runners/README.md).
These values can be specifed using [CI variables](../../ci/variables/README.md): These values can be specified using [CI variables](../../ci/variables/README.md):
- `GITLAB_RUNNER_GITLAB_URL` will be used for `gitlabUrl`. - `GITLAB_RUNNER_GITLAB_URL` will be used for `gitlabUrl`.
- `GITLAB_RUNNER_REGISTRATION_TOKEN` will be used for `runnerRegistrationToken` - `GITLAB_RUNNER_REGISTRATION_TOKEN` will be used for `runnerRegistrationToken`
...@@ -914,7 +914,7 @@ when upgrading the Vault application. ...@@ -914,7 +914,7 @@ when upgrading the Vault application.
To optimally use Vault in a production environment, it's ideal to have a good understanding To optimally use Vault in a production environment, it's ideal to have a good understanding
of the internals of Vault and how to configure it. This can be done by reading the of the internals of Vault and how to configure it. This can be done by reading the
[the Vault documentation](https://www.vaultproject.io/docs/internals/) as well as [the Vault documentation](https://www.vaultproject.io/docs/internals/) as well as
the Vault Helm chart [values.yaml file](https://github.com/hashicorp/vault-helm/blob/v0.3.3/values.yaml). the Vault Helm chart [`values.yaml` file](https://github.com/hashicorp/vault-helm/blob/v0.3.3/values.yaml).
At a minimum you will likely set up: At a minimum you will likely set up:
...@@ -1040,7 +1040,7 @@ elasticStack: ...@@ -1040,7 +1040,7 @@ elasticStack:
Elastic Stack is installed into the `gitlab-managed-apps` namespace of your cluster. Elastic Stack is installed into the `gitlab-managed-apps` namespace of your cluster.
You can check the default [values.yaml](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/elastic_stack/values.yaml) we set for this chart. You can check the default [`values.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/elastic_stack/values.yaml) we set for this chart.
You can customize the installation of Elastic Stack by defining You can customize the installation of Elastic Stack by defining
`.gitlab/managed-apps/elastic-stack/values.yaml` file in your cluster `.gitlab/managed-apps/elastic-stack/values.yaml` file in your cluster
...@@ -1068,7 +1068,7 @@ Crossplane: ...@@ -1068,7 +1068,7 @@ Crossplane:
Crossplane is installed into the `gitlab-managed-apps` namespace of your cluster. Crossplane is installed into the `gitlab-managed-apps` namespace of your cluster.
You can check the default You can check the default
[values.yaml](https://github.com/crossplane/crossplane/blob/master/cluster/charts/crossplane/values.yaml.tmpl) [`values.yaml`](https://github.com/crossplane/crossplane/blob/master/cluster/charts/crossplane/values.yaml.tmpl)
we set for this chart. we set for this chart.
You can customize the installation of Crossplane by defining You can customize the installation of Crossplane by defining
...@@ -1088,7 +1088,7 @@ Fluentd: ...@@ -1088,7 +1088,7 @@ Fluentd:
installed: true installed: true
``` ```
You can also review the default values set for this chart in the [values.yaml](https://github.com/helm/charts/blob/master/stable/fluentd/values.yaml) file. You can also review the default values set for this chart in the [`values.yaml`](https://github.com/helm/charts/blob/master/stable/fluentd/values.yaml) file.
You can customize the installation of Fluentd by defining You can customize the installation of Fluentd by defining
`.gitlab/managed-apps/fluentd/values.yaml` file in your cluster management `.gitlab/managed-apps/fluentd/values.yaml` file in your cluster management
......
...@@ -37,8 +37,8 @@ anything for them to work. ...@@ -37,8 +37,8 @@ anything for them to work.
Explaining [every detail of GitLab CI/CD](../../../ci/yaml/README.md) Explaining [every detail of GitLab CI/CD](../../../ci/yaml/README.md)
and GitLab Runner is out of the scope of this guide, but we'll and GitLab Runner is out of the scope of this guide, but we'll
need to understand just a few things to be able to write our own need to understand just a few things to be able to write our own
`.gitlab-ci.yml` or tweak an existing one. It's an `.gitlab-ci.yml` or tweak an existing one. It's a
[Yaml](https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html) file, [YAML](https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html) file,
with its own syntax. You can always check your CI syntax with with its own syntax. You can always check your CI syntax with
the [GitLab CI/CD Lint Tool](https://gitlab.com/ci/lint). the [GitLab CI/CD Lint Tool](https://gitlab.com/ci/lint).
...@@ -60,7 +60,7 @@ jekyll build ...@@ -60,7 +60,7 @@ jekyll build
### Script ### Script
To transpose this script to Yaml, it would be like this: To transpose this script to YAML, it would be like this:
```yaml ```yaml
script: script:
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment