@@ -1236,7 +1236,7 @@ Open the rails console (`gitlab rails c`) and run the following command to see a
...
@@ -1236,7 +1236,7 @@ Open the rails console (`gitlab rails c`) and run the following command to see a
ApplicationSetting.last.attributes
ApplicationSetting.last.attributes
```
```
Among other attributes, in the output you will notice that all the settings available in the [Elasticsearch Integration page](../../integration/elasticsearch.md), like: `elasticsearch_indexing`, `elasticsearch_url`, `elasticsearch_replicas`, `elasticsearch_pause_indexing`, etc.
Among other attributes, in the output you will notice that all the settings available in the [Elasticsearch Integration page](../../integration/elasticsearch.md), like: `elasticsearch_indexing`, `elasticsearch_url`, `elasticsearch_replicas`, `elasticsearch_pause_indexing`, and so on.
| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user |
| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user |
| `name` | string | yes | The name of the label |
| `name` | string | yes | The name of the label |
| `color` | string | yes | The color of the label given in 6-digit hex notation with leading '#' sign (e.g. #FFAABB) or one of the [CSS color names](https://developer.mozilla.org/en-US/docs/Web/CSS/color_value#Color_keywords) |
| `color` | string | yes | The color of the label given in 6-digit hex notation with leading '#' sign (for example, #FFAABB) or one of the [CSS color names](https://developer.mozilla.org/en-US/docs/Web/CSS/color_value#Color_keywords) |
| `description` | string | no | The description of the label, |
| `description` | string | no | The description of the label, |
```shell
```shell
...
@@ -155,7 +155,7 @@ PUT /groups/:id/labels/:label_id
...
@@ -155,7 +155,7 @@ PUT /groups/:id/labels/:label_id
| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user |
| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user |
| `label_id` | integer or string | yes | The ID or title of a group's label. |
| `label_id` | integer or string | yes | The ID or title of a group's label. |
| `new_name` | string | no | The new name of the label |
| `new_name` | string | no | The new name of the label |
| `color` | string | no | The color of the label given in 6-digit hex notation with leading '#' sign (e.g. #FFAABB) or one of the [CSS color names](https://developer.mozilla.org/en-US/docs/Web/CSS/color_value#Color_keywords) |
| `color` | string | no | The color of the label given in 6-digit hex notation with leading '#' sign (for example, #FFAABB) or one of the [CSS color names](https://developer.mozilla.org/en-US/docs/Web/CSS/color_value#Color_keywords) |
| `description` | string | no | The description of the label. |
| `description` | string | no | The description of the label. |
@@ -1402,11 +1402,11 @@ POST /groups/:id/push_rule
...
@@ -1402,11 +1402,11 @@ POST /groups/:id/push_rule
| `deny_delete_tag` | boolean | no | Deny deleting a tag |
| `deny_delete_tag` | boolean | no | Deny deleting a tag |
| `member_check` | boolean | no | Allows only GitLab users to author commits |
| `member_check` | boolean | no | Allows only GitLab users to author commits |
| `prevent_secrets` | boolean | no | [Files that are likely to contain secrets](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/gitlab/checks/files_denylist.yml) are rejected |
| `prevent_secrets` | boolean | no | [Files that are likely to contain secrets](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/gitlab/checks/files_denylist.yml) are rejected |
| `commit_message_regex` | string | no | All commit messages must match the regular expression provided in this attribute, e.g.`Fixed \d+\..*` |
| `commit_message_regex` | string | no | All commit messages must match the regular expression provided in this attribute, for example,`Fixed \d+\..*` |
| `commit_message_negative_regex` | string | no | Commit messages matching the regular expression provided in this attribute aren't allowed, e.g.`ssh\:\/\/` |
| `commit_message_negative_regex` | string | no | Commit messages matching the regular expression provided in this attribute aren't allowed, for example,`ssh\:\/\/` |
| `branch_name_regex` | string | no | All branch names must match the regular expression provided in this attribute, e.g.`(feature|hotfix)\/*` |
| `branch_name_regex` | string | no | All branch names must match the regular expression provided in this attribute, for example,`(feature|hotfix)\/*` |
| `author_email_regex` | string | no | All commit author emails must match the regular expression provided in this attribute, e.g.`@my-company.com$` |
| `author_email_regex` | string | no | All commit author emails must match the regular expression provided in this attribute, for example,`@my-company.com$` |
| `file_name_regex` | string | no | Filenames matching the regular expression provided in this attribute are **not** allowed, e.g.`(jar|exe)$` |
| `file_name_regex` | string | no | Filenames matching the regular expression provided in this attribute are **not** allowed, for example,`(jar|exe)$` |
| `max_file_size` | integer | no | Maximum file size (MB) allowed |
| `max_file_size` | integer | no | Maximum file size (MB) allowed |
| `commit_committer_check` | boolean | no | Only commits pushed using verified emails are allowed |
| `commit_committer_check` | boolean | no | Only commits pushed using verified emails are allowed |
| `reject_unsigned_commits` | boolean | no | Only commits signed through GPG are allowed |
| `reject_unsigned_commits` | boolean | no | Only commits signed through GPG are allowed |
...
@@ -1449,11 +1449,11 @@ PUT /groups/:id/push_rule
...
@@ -1449,11 +1449,11 @@ PUT /groups/:id/push_rule
| `deny_delete_tag` | boolean | no | Deny deleting a tag |
| `deny_delete_tag` | boolean | no | Deny deleting a tag |
| `member_check` | boolean | no | Restricts commits to be authored by existing GitLab users only |
| `member_check` | boolean | no | Restricts commits to be authored by existing GitLab users only |
| `prevent_secrets` | boolean | no | [Files that are likely to contain secrets](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/gitlab/checks/files_denylist.yml) are rejected |
| `prevent_secrets` | boolean | no | [Files that are likely to contain secrets](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/gitlab/checks/files_denylist.yml) are rejected |
| `commit_message_regex` | string | no | All commit messages must match the regular expression provided in this attribute, e.g.`Fixed \d+\..*` |
| `commit_message_regex` | string | no | All commit messages must match the regular expression provided in this attribute, for example,`Fixed \d+\..*` |
| `commit_message_negative_regex` | string | no | Commit messages matching the regular expression provided in this attribute aren't allowed, e.g.`ssh\:\/\/` |
| `commit_message_negative_regex` | string | no | Commit messages matching the regular expression provided in this attribute aren't allowed, for example,`ssh\:\/\/` |
| `branch_name_regex` | string | no | All branch names must match the regular expression provided in this attribute, e.g.`(feature|hotfix)\/*` |
| `branch_name_regex` | string | no | All branch names must match the regular expression provided in this attribute, for example,`(feature|hotfix)\/*` |
| `author_email_regex` | string | no | All commit author emails must match the regular expression provided in this attribute, e.g.`@my-company.com$` |
| `author_email_regex` | string | no | All commit author emails must match the regular expression provided in this attribute, for example,`@my-company.com$` |
| `file_name_regex` | string | no | Filenames matching the regular expression provided in this attribute are **not** allowed, e.g.`(jar|exe)$` |
| `file_name_regex` | string | no | Filenames matching the regular expression provided in this attribute are **not** allowed, for example,`(jar|exe)$` |
| `max_file_size` | integer | no | Maximum file size (MB) allowed |
| `max_file_size` | integer | no | Maximum file size (MB) allowed |
| `commit_committer_check` | boolean | no | Only commits pushed using verified emails are allowed |
| `commit_committer_check` | boolean | no | Only commits pushed using verified emails are allowed |
| `reject_unsigned_commits` | boolean | no | Only commits signed through GPG are allowed |
| `reject_unsigned_commits` | boolean | no | Only commits signed through GPG are allowed |
| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
| `name` | string | yes | The name of the label |
| `name` | string | yes | The name of the label |
| `color` | string | yes | The color of the label given in 6-digit hex notation with leading '#' sign (e.g. #FFAABB) or one of the [CSS color names](https://developer.mozilla.org/en-US/docs/Web/CSS/color_value#Color_keywords) |
| `color` | string | yes | The color of the label given in 6-digit hex notation with leading '#' sign (for example, #FFAABB) or one of the [CSS color names](https://developer.mozilla.org/en-US/docs/Web/CSS/color_value#Color_keywords) |
| `description` | string | no | The description of the label |
| `description` | string | no | The description of the label |
| `priority` | integer | no | The priority of the label. Must be greater or equal than zero or `null` to remove the priority. |
| `priority` | integer | no | The priority of the label. Must be greater or equal than zero or `null` to remove the priority. |
...
@@ -219,7 +219,7 @@ PUT /projects/:id/labels/:label_id
...
@@ -219,7 +219,7 @@ PUT /projects/:id/labels/:label_id
| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
| `label_id` | integer or string | yes | The ID or title of a group's label. |
| `label_id` | integer or string | yes | The ID or title of a group's label. |
| `new_name` | string | yes if `color` is not provided | The new name of the label |
| `new_name` | string | yes if `color` is not provided | The new name of the label |
| `color` | string | yes if `new_name` is not provided | The color of the label given in 6-digit hex notation with leading '#' sign (e.g. #FFAABB) or one of the [CSS color names](https://developer.mozilla.org/en-US/docs/Web/CSS/color_value#Color_keywords) |
| `color` | string | yes if `new_name` is not provided | The color of the label given in 6-digit hex notation with leading '#' sign (for example, #FFAABB) or one of the [CSS color names](https://developer.mozilla.org/en-US/docs/Web/CSS/color_value#Color_keywords) |
| `description` | string | no | The new description of the label |
| `description` | string | no | The new description of the label |
| `priority` | integer | no | The new priority of the label. Must be greater or equal than zero or `null` to remove the priority. |
| `priority` | integer | no | The new priority of the label. Must be greater or equal than zero or `null` to remove the priority. |
Client applications (e.g. GitLab Rails and Docker CLI) interact with the Container Registry through its [HTTP API](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs/spec/api.md). The most common operations are pushing and pulling images to/from the registry, which require a series of HTTP requests in a specific order. The request flow for these operations is detailed in the [Request flow](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs-gitlab/push-pull-request-flow.md).
Client applications (for example, GitLab Rails and Docker CLI) interact with the Container Registry through its [HTTP API](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs/spec/api.md). The most common operations are pushing and pulling images to/from the registry, which require a series of HTTP requests in a specific order. The request flow for these operations is detailed in the [Request flow](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs-gitlab/push-pull-request-flow.md).
The registry supports multiple [storage backends](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs/configuration.md#storage), including Google Cloud Storage (GCS) which is used for the GitLab.com registry. In the storage backend, images are stored as blobs, deduplicated, and shared across repositories. These are then linked (like a symlink) to each repository that relies on them, giving them access to the central storage location.
The registry supports multiple [storage backends](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs/configuration.md#storage), including Google Cloud Storage (GCS) which is used for the GitLab.com registry. In the storage backend, images are stored as blobs, deduplicated, and shared across repositories. These are then linked (like a symlink) to each repository that relies on them, giving them access to the central storage location.
...
@@ -54,7 +54,7 @@ sequenceDiagram
...
@@ -54,7 +54,7 @@ sequenceDiagram
C->>R: docker login gitlab.example.com
C->>R: docker login gitlab.example.com
R->>C: 401 Unauthorized
R->>C: 401 Unauthorized
Note left of R: The response includes the realm (e.g., https://gitlab.example.com/jwt/auth)<br> from where a token should be obtained
Note left of R: The response includes the realm (for example, https://gitlab.example.com/jwt/auth)<br> from where a token should be obtained
Mid-term, we plan to expand the level of detail the testing pipeline reports back to the Merge Request and expand its scope to cover query testing, too. By doing so, we use our experience from database code reviews and using thin-clone technology and bring this back closer to the GitLab workflow. Instead of reaching out to different tools (`postgres.ai`, `joe`, Slack, plan visualizations etc.) we bring this back to GitLab and working directly on the Merge Request.
Mid-term, we plan to expand the level of detail the testing pipeline reports back to the Merge Request and expand its scope to cover query testing, too. By doing so, we use our experience from database code reviews and using thin-clone technology and bring this back closer to the GitLab workflow. Instead of reaching out to different tools (`postgres.ai`, `joe`, Slack, plan visualizations, and so on) we bring this back to GitLab and working directly on the Merge Request.
Secondly, we plan to cover background migrations testing, too. These are typically data migrations that are scheduled to run over a long period of time. The success of both the scheduling phase and the job execution phase typically depends a lot on data distribution - which only surfaces when running these migrations on actual production data. In order to become confident about a background migration, we plan to provide the following feedback:
Secondly, we plan to cover background migrations testing, too. These are typically data migrations that are scheduled to run over a long period of time. The success of both the scheduling phase and the job execution phase typically depends a lot on data distribution - which only surfaces when running these migrations on actual production data. In order to become confident about a background migration, we plan to provide the following feedback:
1. Scheduling phase - query statistics (for example a histogram of query execution times), job statistics (how many jobs, overall duration etc.), batch sizes.
1. Scheduling phase - query statistics (for example a histogram of query execution times), job statistics (how many jobs, overall duration, and so on), batch sizes.
1. Execution phase - using a few instances of a job as examples, we execute those to gather query and runtime statistics.
1. Execution phase - using a few instances of a job as examples, we execute those to gather query and runtime statistics.
### Long-term - incorporate into GitLab product
### Long-term - incorporate into GitLab product
...
@@ -114,7 +114,7 @@ At the core of this problem lies the concern about executing (potentially arbitr
...
@@ -114,7 +114,7 @@ At the core of this problem lies the concern about executing (potentially arbitr
An alternative approach we have discussed and abandoned is to "scrub" and anonymize production data. The idea is to remove any sensitive data from the database and use the resulting dataset for database testing. This has a lot of downsides which led us to abandon the idea:
An alternative approach we have discussed and abandoned is to "scrub" and anonymize production data. The idea is to remove any sensitive data from the database and use the resulting dataset for database testing. This has a lot of downsides which led us to abandon the idea:
- Anonymization is complex by nature - it is a hard problem to call a "scrubbed clone" actually safe to work with in public. Different data types may require different anonymization techniques (e.g. anonymizing sensitive information inside a JSON field) and only focusing on one attribute at a time does not guarantee that a dataset is fully anonymized (for example join attacks or using timestamps in conjunction to public profiles/projects to de-anonymize users by there activity).
- Anonymization is complex by nature - it is a hard problem to call a "scrubbed clone" actually safe to work with in public. Different data types may require different anonymization techniques (for example, anonymizing sensitive information inside a JSON field) and only focusing on one attribute at a time does not guarantee that a dataset is fully anonymized (for example join attacks or using timestamps in conjunction to public profiles/projects to de-anonymize users by there activity).
- Anonymization requires an additional process to keep track and update the set of attributes considered as sensitive, ongoing maintenance and security reviews every time the database schema changes.
- Anonymization requires an additional process to keep track and update the set of attributes considered as sensitive, ongoing maintenance and security reviews every time the database schema changes.
- Annotating data as "sensitive" is error prone, with the wrong anonymization approach used for a data type or one sensitive attribute accidentally not marked as such possibly leading to a data breach.
- Annotating data as "sensitive" is error prone, with the wrong anonymization approach used for a data type or one sensitive attribute accidentally not marked as such possibly leading to a data breach.
- Scrubbing not only removes sensitive data, but it also changes data distribution, which greatly affects performance of migrations and queries.
- Scrubbing not only removes sensitive data, but it also changes data distribution, which greatly affects performance of migrations and queries.
@@ -39,7 +39,7 @@ Content image resizing is a more complex problem to tackle. There are no set siz
...
@@ -39,7 +39,7 @@ Content image resizing is a more complex problem to tackle. There are no set siz
- Extract first image of GIF's so we can prevent from loading 10MB pixels
- Extract first image of GIF's so we can prevent from loading 10MB pixels
- Check Device Pixel Ratio to deliver nice images on High DPI screens
- Check Device Pixel Ratio to deliver nice images on High DPI screens
- Progressive image loading, similar to what is described in [this article about how to build a progressive image loader](https://www.sitepoint.com/how-to-build-your-own-progressive-image-loader/)
- Progressive image loading, similar to what is described in [this article about how to build a progressive image loader](https://www.sitepoint.com/how-to-build-your-own-progressive-image-loader/)
- Resizing recommendations (size, clarity, etc.)
- Resizing recommendations (size, clarity, and so on)
- Storage
- Storage
The MVC Avatar resizing implementation is integrated into Workhorse. With the extra requirements for content image resizing, this may require further use of GraphicsMagik (GM) or a similar library and breaking it out of Workhorse.
The MVC Avatar resizing implementation is integrated into Workhorse. With the extra requirements for content image resizing, this may require further use of GraphicsMagik (GM) or a similar library and breaking it out of Workhorse.