Skip to content

G
gitlab-ce
Commit f4a1dcbe authored by Marcel Amirault's avatar Marcel Amirault Committed by Achilleas Pipinellis
Browse files
Options

Docs: Merge Misc EE doc/administration files and dirs to CE

parent 9b06dffc
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 893 additions and 15 deletions
doc/administration/audit_events.md 0 → 100644
View file @ f4a1dcbe
---
last_updated: 2019-02-04
---
# Audit Events **[STARTER]**
GitLab offers a way to view the changes made within the GitLab server for owners and administrators on a [paid plan][ee].
GitLab system administrators can also take advantage of the logs located on the
filesystem, see [the logs system documentation](logs.md) for more details.
## Overview
**Audit Events** is a tool for GitLab owners and administrators to be
able to track important events such as who performed certain actions and the
time they happened. These actions could be, for example, change a user
permission level, who added a new user, or who removed a user.
## Use-cases
- Check who was the person who changed the permission level of a particular
user for a project in GitLab.
- Use it to track which users have access to a certain group of projects
in GitLab, and who gave them that permission level.
## List of events
There are two kinds of events logged:
- Events scoped to the group or project, used by group / project managers
to look up who made what change.
- Instance events scoped to the whole GitLab instance, used by your Compliance team to
perform formal audits.
### Group events **[STARTER]**
NOTE: **Note:**
You need Owner [permissions] to view the group Audit Events page.
To view a group's audit events, navigate to **Group > Settings > Audit Events**.
From there, you can see the following actions:
- Group name/path changed
- Group repository size limit changed
- Group created/deleted
- Group changed visibility
- User was added to group and with which [permissions]
- Permissions changes of a user assigned to a group
- Removed user from group
- Project added to group and with which visibility level
- Project removed from group
- [Project shared with group](../user/project/members/share_project_with_groups.md)
and with which [permissions]
- Removal of a previously shared group with a project
- LFS enabled/disabled
- Shared runners minutes limit changed
- Membership lock enabled/disabled
- Request access enabled/disabled
- 2FA enforcement/grace period changed
- Roles allowed to create project changed
### Project events **[STARTER]**
NOTE: **Note:**
You need Maintainer [permissions] or higher to view the project Audit Events page.
To view a project's audit events, navigate to **Project > Settings > Audit Events**.
From there, you can see the following actions:
- Added/removed deploy keys
- Project created/deleted/renamed/moved(transferred)/changed path
- Project changed visibility level
- User was added to project and with which [permissions]
- Permission changes of a user assigned to a project
- User was removed from project
### Instance events **[PREMIUM ONLY]**
> [Introduced][ee-2336] in [GitLab Premium][ee] 9.3.
Server-wide audit logging introduces the ability to observe user actions across
the entire instance of your GitLab server, making it easy to understand who
changed what and when for audit purposes.
To view the server-wide admin log, visit **Admin Area > Monitoring > Audit Log**.
In addition to the group and project events, the following user actions are also
recorded:
- Failed Logins
- Sign-in events and the authentication type (standard, LDAP, OmniAuth, etc.)
- Added SSH key
- Added/removed email
- Changed password
- Ask for password reset
- Grant OAuth access
It is possible to filter particular actions by choosing an audit data type from
the filter drop-down. You can further filter by specific group, project or user
(for authentication events).
![audit log](audit_log.png)
### Missing events
Some events are not being tracked in Audit Events. Please see the following
epics for more detail on which events are not being tracked and our progress
on adding these events into GitLab:
- [Project settings and activity](https://gitlab.com/groups/gitlab-org/-/epics/474)
- [Group settings and activity](https://gitlab.com/groups/gitlab-org/-/epics/475)
- [Instance-level settings and activity](https://gitlab.com/groups/gitlab-org/-/epics/476)
[ee-2336]: https://gitlab.com/gitlab-org/gitlab-ee/issues/2336
[ee]: https://about.gitlab.com/pricing/
[permissions]: ../user/permissions.md
doc/administration/auditor_users.md 0 → 100644
View file @ f4a1dcbe
# Auditor users **[PREMIUM ONLY]**
>[Introduced][ee-998] in [GitLab Premium][eep] 8.17.
Auditor users are given read-only access to all projects, groups, and other
resources on the GitLab instance.
## Overview
Auditor users can have full access to their own resources (projects, groups,
snippets, etc.), and read-only access to **all** other resources, except the
Admin area. To put another way, they are just regular users (who can be added
to projects, create personal snippets, create milestones on their groups, etc.)
who also happen to have read-only access to all projects on the system that
they haven't been explicitly [given access][permissions] to.
The Auditor role is _not_ a read-only version of the Admin role. Auditor users
will not be able to access the project/group settings pages, or the Admin Area.
To sum up, assuming you have logged-in as an Auditor user:
- For a project the Auditor is not member of, the Auditor should have
read-only access. If the project is public or internal, they would have the
same access as the users that are not members of that project/group.
- For a project the Auditor owns, the Auditor should have full access to
everything.
- For a project the Auditor has been added to as a member, the Auditor should
have the same access as the [permissions] they were given to. For example, if
they were added as a Developer, they could then push commits or comment on
issues.
- The Auditor cannot view the Admin area, or perform any admin actions.
For more information about what an Auditor can or can't do, see the
[Permissions and restrictions of an Auditor user](#permissions-and-restrictions-of-an-auditor-user)
section.
## Use cases
1. Your compliance department wants to run tests against the entire GitLab base
to ensure users are complying with password, credit card, and other sensitive
data policies. With Auditor users, this can be achieved very easily without
resulting to tactics like giving a user admin rights or having to use the API
to add them to all projects.
1. If particular users need visibility or access to most of all projects in
your GitLab instance, instead of manually adding the user to all projects,
you can simply create an Auditor user and share the credentials with those
that you want to grant access to.
## Adding an Auditor user
1. Create a new user or edit an existing one by navigating to
**Admin Area > Users**. You will find the option of the access level under
the 'Access' section.
![Admin Area Form](auditor_access_form.png)
1. Click **Save changes** or **Create user** for the changes to take effect.
To revoke the Auditor permissions from a user, simply make them a Regular user
following the same steps as above.
## Permissions and restrictions of an Auditor user
An Auditor user should be able to access all projects and groups of a GitLab
instance, with the following permissions/restrictions:
- Has read-only access to the API
- Can access projects that are:
- Private
- Public
- Internal
- Can read all files in a repository
- Can read issues / MRs
- Can read project snippets
- Cannot be Admin and Auditor at the same time
- Cannot access the Admin area
- In a group / project they're not a member of:
- Cannot access project settings
- Cannot access group settings
- Cannot commit to repository
- Cannot create / comment on issues / MRs
- Cannot create/modify files from the Web UI
- Cannot merge a merge request
- Cannot create project snippets
[ee-998]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/998
[eep]: https://about.gitlab.com/pricing/
[permissions]: ../user/permissions.md
doc/administration/compliance.md
View file @ f4a1dcbe
......@@ -11,8 +11,8 @@ GitLab’s [security features](../security/README.md) may also help you meet rel
|**[Enforce TOS acceptance](../user/admin_area/settings/terms.md)**<br>Enforce your users accepting new terms of service by blocking GitLab traffic.|Core+||
|**[Email all users of a project, group, or entire server](../user/admin_area/settings/terms.md)**<br>An admin can email groups of users based on project or group membership, or email everyone using the GitLab instance. This is great for scheduled maintenance or upgrades.|Starter+||
|**[Omnibus package supports log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-forwarding)**<br>Forward your logs to a central system.|Starter+||
|**[Lock project membership to group](https://docs.gitlab.com/ee/user/group/index.html#member-lock-starter)**<br>Group owners can prevent new members from being added to projects within a group.|Starter+|✓|
|**[Lock project membership to group](../user/group/index.md#member-lock-starter)**<br>Group owners can prevent new members from being added to projects within a group.|Starter+|✓|
|**[LDAP group sync](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync)**<br>GitLab Enterprise Edition gives admins the ability to automatically sync groups and manage SSH keys, permissions, and authentication, so you can focus on building your product, not configuring your tools.|Starter+||
|**[LDAP group sync filters](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync)**<br>GitLab Enterprise Edition Premium gives more flexibility to synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions.|Premium+||
|**[Audit logs](https://docs.gitlab.com/ee/administration/audit_events.html)**<br>To maintain the integrity of your code, GitLab Enterprise Edition Premium gives admins the ability to view any modifications made within the GitLab server in an advanced audit log system, so you can control, analyze and track every change.|Premium+||
|**[Auditor users](https://docs.gitlab.com/ee/administration/auditor_users.html)**<br>Auditor users are users who are given read-only access to all projects, groups, and other resources on the GitLab instance.|Premium+||
\ No newline at end of file
|**[Audit logs](audit_events.md)**<br>To maintain the integrity of your code, GitLab Enterprise Edition Premium gives admins the ability to view any modifications made within the GitLab server in an advanced audit log system, so you can control, analyze and track every change.|Premium+||
|**[Auditor users](auditor_users.md)**<br>Auditor users are users who are given read-only access to all projects, groups, and other resources on the GitLab instance.|Premium+||
doc/administration/custom_hooks.md
View file @ f4a1dcbe
......@@ -8,6 +8,9 @@ have filesystem access. For a user configurable Git hook interface, see
[Push Rules](https://docs.gitlab.com/ee/push_rules/push_rules.html),
available in GitLab Enterprise Edition.
NOTE: **Note:**
Custom Git hooks won't be replicated to secondary nodes if you use [GitLab Geo][gitlab-geo]
Git natively supports hooks that are executed on different actions.
Examples of server-side git hooks include pre-receive, post-receive, and update.
See [Git SCM Server-Side Hooks][hooks] for more information about each hook type.
......@@ -48,7 +51,7 @@ To create a Git hook that applies to all of your repositories in
your instance, set a global Git hook. Since all the repositories' `hooks`
directories are symlinked to gitlab-shell's `hooks` directory, adding any hook
to the gitlab-shell `hooks` directory will also apply it to all repositories. Follow
the steps below to properly set up a custom hook all for repositories:
the steps below to properly set up a custom hook for all repositories:
1. On the GitLab server, navigate to the configured custom hook directory. The
default is in the gitlab-shell directory. The gitlab-shell `hook` directory
......@@ -120,5 +123,6 @@ exit 1
[CI]: ../ci/README.md
[hooks]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#Server-Side-Hooks
[webhooks]: ../user/project/integrations/webhooks.md
[gitlab-geo]: https://docs.gitlab.com/ee/administration/geo/replication/index.html
[5073]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5073
[93]: https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/93
doc/administration/database_load_balancing.md 0 → 100644
View file @ f4a1dcbe
This diff is collapsed.
doc/administration/high_availability/README.md
View file @ f4a1dcbe
......@@ -42,8 +42,6 @@ Follow the steps below to configure an active/active setup:
1. [Configure the GitLab application servers](gitlab.md)
1. [Configure the load balancers](load_balancer.md)
![Active/Active HA Diagram](../img/high_availability/active-active-diagram.png)
### Active/Passive
For pure high-availability/failover with no scaling you can use an
......@@ -59,5 +57,3 @@ or in different cloud availability zones.
root cause is identified as DRBD, we will not troubleshoot further.
Components/Servers Required: 2 servers/virtual machines (one active/one passive)
![Active/Passive HA Diagram](../img/high_availability/active-passive-diagram.png)
doc/administration/incoming_email.md
View file @ f4a1dcbe
......@@ -10,6 +10,8 @@ GitLab has several features based on receiving incoming emails:
- [New merge request by email](../user/project/merge_requests/index.md#create-new-merge-requests-by-email):
allow GitLab users to create a new merge request by sending an email to a
user-specific email address.
- [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html): provide e-mail support to
your customers through GitLab. **[PREMIUM]**
## Requirements
......
doc/administration/index.md
View file @ f4a1dcbe
......@@ -32,8 +32,14 @@ Learn how to install, configure, update, and maintain your GitLab instance.
### Installing GitLab
- [Install](../install/README.md): Requirements, directory structures, and installation methods.
- [Database load balancing](database_load_balancing.md): Distribute database queries among multiple database servers. **[STARTER ONLY]**
- [Omnibus support for external MySQL DB](https://docs.gitlab.com/omnibus/settings/database.html#using-a-mysql-database-management-server-enterprise-edition-only): Omnibus package supports configuring an external MySQL database. **[STARTER ONLY]**
- [Omnibus support for log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only) **[STARTER ONLY]**
- [High Availability](high_availability/README.md): Configure multiple servers for scaling or high availability.
- [High Availability on AWS](../university/high-availability/aws/README.md): Set up GitLab HA on Amazon AWS.
- [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html): Replicate your GitLab instance to other geographic locations as a read-only fully operational version. **[PREMIUM ONLY]**
- [Disaster Recovery](https://docs.gitlab.com/ee/administration/geo/disaster_recovery/index.html): Quickly fail-over to a different site with minimal effort in a disaster situation. **[PREMIUM ONLY]**
- [Pivotal Tile](https://docs.gitlab.com/ee/install/pivotal/index.html): Deploy GitLab as a pre-configured appliance using Ops Manager (BOSH) for Pivotal Cloud Foundry. **[PREMIUM ONLY]**
### Configuring GitLab
......@@ -44,8 +50,8 @@ Learn how to install, configure, update, and maintain your GitLab instance.
- [Global user settings](user_settings.md): Configure instance-wide user permissions.
- [Polling](polling.md): Configure how often the GitLab UI polls for updates.
- [GitLab Pages configuration](pages/index.md): Enable and configure GitLab Pages.
- [GitLab Pages configuration for GitLab source installations](pages/source.md): Enable and configure GitLab Pages on
[source installations](../install/installation.md#installation-from-source).
- [GitLab Pages configuration for GitLab source installations](pages/source.md): Enable and configure GitLab Pages on [source installations](../install/installation.md#installation-from-source).
- [Uploads configuration](uploads.md): Configure GitLab uploads storage.
- [Environment variables](environment_variables.md): Supported environment variables that can be used to override their defaults values in order to configure GitLab.
- [Plugins](plugins.md): With custom plugins, GitLab administrators can introduce custom integrations without modifying GitLab's source code.
- [Enforcing Terms of Service](../user/admin_area/settings/terms.md)
......@@ -54,6 +60,9 @@ Learn how to install, configure, update, and maintain your GitLab instance.
- [Diff limits](../user/admin_area/diff_limits.md): Configure the diff rendering size limits of branch comparison pages.
- [Merge request diffs storage](merge_request_diffs.md): Configure merge requests diffs external storage.
- [Broadcast Messages](../user/admin_area/broadcast_messages.md): Send messages to GitLab users through the UI.
- [Elasticsearch](https://docs.gitlab.com/ee/integration/elasticsearch.html): Enable Elasticsearch to empower GitLab's Advanced Global Search. Useful when you deal with a huge amount of data. **[STARTER ONLY]**
- [External Classification Policy Authorization](../user/admin_area/settings/external_authorization.md) **[PREMIUM ONLY]**
- [Upload a license](https://docs.gitlab.com/ee/user/admin_area/license.html): Upload a license to unlock features that are in paid tiers of GitLab. **[STARTER ONLY]**
- [Admin Area](../user/admin_area/index.md): for self-managed instance-wide configuration and maintenance.
#### Customizing GitLab's appearance
......@@ -63,6 +72,7 @@ Learn how to install, configure, update, and maintain your GitLab instance.
- [Branded login page](../customization/branded_login_page.md): Customize the login page with your own logo, title, and description.
- [Welcome message](../customization/welcome_message.md): Add a custom welcome message to the sign-in page.
- ["New Project" page](../customization/new_project_page.md): Customize the text to be displayed on the page that opens whenever your users create a new project.
- [Additional custom email text](https://docs.gitlab.com/ee/user/admin_area/settings/email.html#custom-additional-text): Add additional custom text to emails sent from GitLab. **[PREMIUM ONLY]**
### Maintaining GitLab
......@@ -96,7 +106,16 @@ Learn how to install, configure, update, and maintain your GitLab instance.
- [Libravatar](../customization/libravatar.md): Use Libravatar instead of Gravatar for user avatars.
- [Sign-up restrictions](../user/admin_area/settings/sign_up_restrictions.md): block email addresses of specific domains, or whitelist only specific domains.
- [Access restrictions](../user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols): Define which Git access protocols can be used to talk to GitLab (SSH, HTTP, HTTPS).
- [Authentication and Authorization](auth/README.md): Configure external authentication with LDAP, SAML, CAS and additional providers. See also other [authentication](../topics/authentication/index.md#gitlab-administrators) topics (for example, enforcing 2FA).
- [Authentication and Authorization](auth/README.md): Configure external authentication with LDAP, SAML, CAS and additional providers.
- [Sync LDAP](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html) **[STARTER ONLY]**
- [Kerberos authentication](https://docs.gitlab.com/ee/integration/kerberos.html) **[STARTER ONLY]**
- See also other [authentication](../topics/authentication/index.md#gitlab-administrators) topics (for example, enforcing 2FA).
- [Email users](https://docs.gitlab.com/ee/tools/email.html): Email GitLab users from within GitLab. **[STARTER ONLY]**
- [User Cohorts](../user/admin_area/user_cohorts.md): Display the monthly cohorts of new users and their activities over time.
- [Audit logs and events](audit_events.md): View the changes made within the GitLab server for:
- Groups and projects. **[STARTER]**
- Instances. **[PREMIUM ONLY]**
- [Auditor users](auditor_users.md): Users with read-only access to all projects, groups, and other resources on the GitLab instance. **[PREMIUM ONLY]**
- [Incoming email](incoming_email.md): Configure incoming emails to allow
users to [reply by email], create [issues by email] and
[merge requests by email], and to enable [Service Desk].
......@@ -108,6 +127,7 @@ Learn how to install, configure, update, and maintain your GitLab instance.
[reply by email]: reply_by_email.md
[issues by email]: ../user/project/issues/create_new_issue.md#new-issue-via-email
[merge requests by email]: ../user/project/merge_requests/index.md#create-new-merge-requests-by-email
[Service Desk]: https://docs.gitlab.com/ee/user/project/service_desk.html
## Project settings
......@@ -116,13 +136,15 @@ Learn how to install, configure, update, and maintain your GitLab instance.
- [Gitaly](gitaly/index.md): Configuring Gitaly, GitLab's Git repository storage service.
- [Default labels](../user/admin_area/labels.html): Create labels that will be automatically added to every new project.
- [Restrict the use of public or internal projects](../public_access/public_access.md#restricting-the-use-of-public-or-internal-projects): Restrict the use of visibility levels for users when they create a project or a snippet.
- [Custom project templates](https://docs.gitlab.com/ee/user/admin_area/custom_project_templates.html): Configure a set of projects to be used as custom templates when creating a new project. **[PREMIUM ONLY]**
- [Custom project templates](../user/admin_area/custom_project_templates.md): Configure a set of projects to be used as custom templates when creating a new project. **[PREMIUM ONLY]**
- [Packages](packages.md): Enable GitLab to act as a Maven repository or NPM registry. **[PREMIUM ONLY]**
### Repository settings
- [Repository checks](repository_checks.md): Periodic Git repository checks.
- [Repository storage paths](repository_storage_paths.md): Manage the paths used to store repositories.
- [Repository storage rake tasks](raketasks/storage.md): A collection of rake tasks to list and migrate existing projects and attachments associated with it from Legacy storage to Hashed storage.
- [Limit repository size](https://docs.gitlab.com/ee/user/admin_area/settings/account_and_limit_settings.html): Set a hard limit for your repositories' size. **[STARTER ONLY]**
## Continuous Integration settings
......@@ -159,6 +181,10 @@ Learn how to install, configure, update, and maintain your GitLab instance.
- [Request Profiling](monitoring/performance/request_profiling.md): Get a detailed profile on slow requests.
- [Performance Bar](monitoring/performance/performance_bar.md): Get performance information for the current page.
## Analytics
- [Pseudonymizer](pseudonymizer.md): Export data from GitLab's database to CSV files in a secure way. **[ULTIMATE]**
## Troubleshooting
- [Debugging tips](troubleshooting/debug.md): Tips to debug problems when things go wrong
......
doc/administration/instance_review.md 0 → 100644
View file @ f4a1dcbe
# Instance Review **[CORE ONLY]**
> [Introduced][6995] in [GitLab Core][ee] 11.3.
If you are running a medium size instance of GitLab Core edition you are qualified for a free Instance Review. You can find the button in the User menu.
![Instance Review button](img/instance_review_button.png)
When you click the button you will be redirected to a form with prefilled data obtained from your instance.
Once you submit the data to GitLab Inc. you can see the initial report.
Additionally you will be contacted by our team for further review which should help you to improve your usage of GitLab.
[6995]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6995
[ee]: https://about.gitlab.com/pricing/
doc/administration/maven_packages.md 0 → 100644
View file @ f4a1dcbe
---
redirect_to: 'packages.md'
---
This document was moved to [another location](packages.md).
doc/administration/maven_repository.md 0 → 100644
View file @ f4a1dcbe
---
redirect_to: 'packages.md'
---
This document was moved to [another location](packages.md).
doc/administration/npm_registry.md 0 → 100644
View file @ f4a1dcbe
---
redirect_to: 'packages.md'
---
This document was moved to [another location](packages.md).
doc/administration/packages.md 0 → 100644
View file @ f4a1dcbe
# GitLab Packages administration **[PREMIUM ONLY]**
GitLab Packages allows organizations to utilize GitLab as a private repository
for a variety of common package managers. Users are able to build and publish
packages, which can be easily consumed as a dependency in downstream projects.
The Packages feature allows GitLab to act as a repository for the following:
| Software repository | Description | Available in GitLab version |
| ------------------- | ----------- | --------------------------- |
| [Maven Repository](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html) | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ |
| [NPM Registry](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html) | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ |
Don't you see your package management system supported yet?
Please consider contributing
to GitLab. This [development documentation](https://docs.gitlab.com/ee/development/packages.html) will guide you through the process.
## Enabling the Packages feature
NOTE: **Note:**
After the Packages feature is enabled, the repositories are available
for all new projects by default. To enable it for existing projects, users will
have to explicitly do so in the project's settings.
To enable the Packages feature:
**Omnibus GitLab installations**
1. Edit `/etc/gitlab/gitlab.rb` and add the following line:
```ruby
gitlab_rails['packages_enabled'] = true
```
1. Save the file and [reconfigure GitLab][] for the changes to take effect.
**Installations from source**
1. After the installation is complete, you will have to configure the `packages`
section in `config/gitlab.yml`. Set to `true` to enable it:
```yaml
packages:
enabled: true
```
1. [Restart GitLab] for the changes to take effect.
## Changing the storage path
By default, the packages are stored locally, but you can change the default
local location or even use object storage.
### Changing the local storage path
The packages for Omnibus GitLab installations are stored under
`/var/opt/gitlab/gitlab-rails/shared/packages/` and for source
installations under `shared/packages/` (relative to the git homedir).
To change the local storage path:
**Omnibus GitLab installations**
1. Edit `/etc/gitlab/gitlab.rb` and add the following line:
```ruby
gitlab_rails['packages_storage_path'] = "/mnt/packages"
```
1. Save the file and [reconfigure GitLab][] for the changes to take effect.
**Installations from source**
1. Edit the `packages` section in `config/gitlab.yml`:
```yaml
packages:
enabled: true
storage_path: shared/packages
```
1. [Restart GitLab] for the changes to take effect.
### Using object storage
Instead of relying on the local storage, you can use an object storage to
upload packages:
**Omnibus GitLab installations**
1. Edit `/etc/gitlab/gitlab.rb` and add the following lines (uncomment where
necessary):
```ruby
gitlab_rails['packages_enabled'] = true
gitlab_rails['packages_storage_path'] = "/var/opt/gitlab/gitlab-rails/shared/packages"
gitlab_rails['packages_object_store_enabled'] = true
gitlab_rails['packages_object_store_remote_directory'] = "packages" # The bucket name.
gitlab_rails['packages_object_store_direct_upload'] = false # Use Object Storage directly for uploads instead of background uploads if enabled (Default: false).
gitlab_rails['packages_object_store_background_upload'] = true # Temporary option to limit automatic upload (Default: true).
gitlab_rails['packages_object_store_proxy_download'] = false # Passthrough all downloads via GitLab instead of using Redirects to Object Storage.
gitlab_rails['packages_object_store_connection'] = {
##
## If the provider is AWS S3, uncomment the following
##
#'provider' => 'AWS',
#'region' => 'eu-west-1',
#'aws_access_key_id' => 'AWS_ACCESS_KEY_ID',
#'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY',
##
## If the provider is other than AWS (an S3-compatible one), uncomment the following
##
#'host' => 's3.amazonaws.com',
#'aws_signature_version' => 4 # For creation of signed URLs. Set to 2 if provider does not support v4.
#'endpoint' => 'https://s3.amazonaws.com' # Useful for S3-compliant services such as DigitalOcean Spaces.
#'path_style' => false # If true, use 'host/bucket_name/object' instead of 'bucket_name.host/object'.
}
```
1. Save the file and [reconfigure GitLab][] for the changes to take effect.
**Installations from source**
1. Edit the `packages` section in `config/gitlab.yml` (uncomment where necessary):
```yaml
packages:
enabled: true
##
## The location where build packages are stored (default: shared/packages).
##
#storage_path: shared/packages
object_store:
enabled: false
remote_directory: packages # The bucket name.
#direct_upload: false # Use Object Storage directly for uploads instead of background uploads if enabled (Default: false).
#background_upload: true # Temporary option to limit automatic upload (Default: true).
#proxy_download: false # Passthrough all downloads via GitLab instead of using Redirects to Object Storage.
connection:
##
## If the provider is AWS S3, uncomment the following
##
#provider: AWS
#region: us-east-1
#aws_access_key_id: AWS_ACCESS_KEY_ID
#aws_secret_access_key: AWS_SECRET_ACCESS_KEY
##
## If the provider is other than AWS (an S3-compatible one), uncomment the following
##
#host: 's3.amazonaws.com' # default: s3.amazonaws.com.
#aws_signature_version: 4 # For creation of signed URLs. Set to 2 if provider does not support v4.
#endpoint: 'https://s3.amazonaws.com' # Useful for S3-compliant services such as DigitalOcean Spaces.
#path_style: false # If true, use 'host/bucket_name/object' instead of 'bucket_name.host/object'.
```
1. [Restart GitLab] for the changes to take effect.
### Migrating local packages to object storage
After [configuring the object storage](#using-object-storage), you may use the
following task to migrate existing packages from the local storage to the remote one.
The processing will be done in a background worker and requires **no downtime**.
For Omnibus GitLab:
```sh
sudo gitlab-rake "gitlab:packages:migrate"
```
For installations from source:
```bash
RAILS_ENV=production sudo -u git -H bundle exec rake gitlab:packages:migrate
```
[reconfigure gitlab]: restart_gitlab.md#omnibus-gitlab-reconfigure "How to reconfigure Omnibus GitLab"
[restart gitlab]: restart_gitlab.md#omnibus-gitlab-reconfigure "How to reconfigure Omnibus GitLab"
doc/administration/pseudonymizer.md 0 → 100644
View file @ f4a1dcbe
# Pseudonymizer **[ULTIMATE]**
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/5532) in [GitLab Ultimate][ee] 11.1.
As GitLab's database hosts sensitive information, using it unfiltered for analytics
implies high security requirements. To help alleviate this constraint, the Pseudonymizer
service is used to export GitLab's data in a pseudonymized way.
CAUTION: **Warning:**
This process is not impervious. If the source data is available, it's possible for
a user to correlate data to the pseudonymized version.
The Pseudonymizer currently uses `HMAC(SHA256)` to mutate fields that shouldn't
be textually exported. This ensures that:
- the end-user of the data source cannot infer/revert the pseudonymized fields
- the referential integrity is maintained
## Configuration
To configure the pseudonymizer, you need to:
- Provide a manifest file that describes which fields should be included or
pseudonymized ([example `manifest.yml` file](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/config/pseudonymizer.yml)).
A default manifest is provided with the GitLab installation. Using a relative file path will be resolved from the Rails root.
Alternatively, you can use an absolute file path.
- Use an object storage and specify the connection parameters in the `pseudonymizer.upload.connection` configuration option.
**For Omnibus installations:**
1. Edit `/etc/gitlab/gitlab.rb` and add the following lines by replacing with
the values you want:
```ruby
gitlab_rails['pseudonymizer_manifest'] = 'config/pseudonymizer.yml'
gitlab_rails['pseudonymizer_upload_remote_directory'] = 'gitlab-elt' # bucket name
gitlab_rails['pseudonymizer_upload_connection'] = {
'provider' => 'AWS',
'region' => 'eu-central-1',
'aws_access_key_id' => 'AWS_ACCESS_KEY_ID',
'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY'
}
```
NOTE: **Note:**
If you are using AWS IAM profiles, be sure to omit the AWS access key and secret access key/value pairs.
```ruby
gitlab_rails['pseudonymizer_upload_connection'] = {
'provider' => 'AWS',
'region' => 'eu-central-1',
'use_iam_profile' => true
}
```
1. Save the file and [reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure)
for the changes to take effect.
---
**For installations from source:**
1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following
lines:
```yaml
pseudonymizer:
manifest: config/pseudonymizer.yml
upload:
remote_directory: 'gitlab-elt' # bucket name
connection:
provider: AWS
aws_access_key_id: AWS_ACCESS_KEY_ID
aws_secret_access_key: AWS_SECRET_ACCESS_KEY
region: eu-central-1
```
1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source)
for the changes to take effect.
## Usage
You can optionally run the pseudonymizer using the following environment variables:
- `PSEUDONYMIZER_OUTPUT_DIR` - where to store the output CSV files (defaults to `/tmp`)
- `PSEUDONYMIZER_BATCH` - the batch size when querying the DB (defaults to `100000`)
```bash
## Omnibus
sudo gitlab-rake gitlab:db:pseudonymizer
## Source
sudo -u git -H bundle exec rake gitlab:db:pseudonymizer RAILS_ENV=production
```
This will produce some CSV files that might be very large, so make sure the
`PSEUDONYMIZER_OUTPUT_DIR` has sufficient space. As a rule of thumb, at least
10% of the database size is recommended.
After the pseudonymizer has run, the output CSV files should be uploaded to the
configured object storage and deleted from the local disk.
[ee]: https://about.gitlab.com/pricing/
doc/administration/raketasks/geo.md 0 → 100644
View file @ f4a1dcbe
# Geo Rake Tasks **[PREMIUM ONLY]**
## Git housekeeping
There are few tasks you can run to schedule a git housekeeping to start at the
next repository sync in a **Secondary node**:
### Incremental Repack
This is equivalent of running `git repack -d` on a _bare_ repository.
**Omnibus Installation**
```
sudo gitlab-rake geo:git:housekeeping:incremental_repack
```
**Source Installation**
```bash
sudo -u git -H bundle exec rake geo:git:housekeeping:incremental_repack RAILS_ENV=production
```
### Full Repack
This is equivalent of running `git repack -d -A --pack-kept-objects` on a
_bare_ repository which will optionally, write a reachability bitmap index
when this is enabled in GitLab.
**Omnibus Installation**
```
sudo gitlab-rake geo:git:housekeeping:full_repack
```
**Source Installation**
```bash
sudo -u git -H bundle exec rake geo:git:housekeeping:full_repack RAILS_ENV=production
```
### GC
This is equivalent of running `git gc` on a _bare_ repository, optionally writing
a reachability bitmap index when this is enabled in GitLab.
**Omnibus Installation**
```
sudo gitlab-rake geo:git:housekeeping:gc
```
**Source Installation**
```bash
sudo -u git -H bundle exec rake geo:git:housekeeping:gc RAILS_ENV=production
```
doc/administration/raketasks/storage.md
View file @ f4a1dcbe
......@@ -43,6 +43,9 @@ If you find it necessary, you can run this migration script again to schedule mi
Any error or warning will be logged in Sidekiq's log file.
NOTE: **Note:**
If Geo is enabled, each project that is successfully migrated generates an event to replicate the changes on any **secondary** nodes.
You only need the `gitlab:storage:migrate_to_hashed` rake task to migrate your repositories, but we have additional
commands below that helps you inspect projects and attachments in both legacy and hashed storage.
......
doc/administration/repository_storage_types.md
View file @ f4a1dcbe
......@@ -47,15 +47,15 @@ Any change in the URL will need to be reflected on disk (when groups / users or
projects are renamed). This can add a lot of load in big installations,
especially if using any type of network based filesystem.
For GitLab Geo in particular: Geo does work with legacy storage, but in some
CAUTION: **Caution:**
For Geo in particular: Geo does work with legacy storage, but in some
edge cases due to race conditions it can lead to errors when a project is
renamed multiple times in short succession, or a project is deleted and
recreated under the same name very quickly. We expect these race events to be
rare, and we have not observed a race condition side-effect happening yet.
This pattern also exists in other objects stored in GitLab, like issue
Attachments, GitLab Pages artifacts, Docker Containers for the integrated
Registry, etc.
Registry, etc. Hashed storage is a requirement for Geo.
## Hashed Storage
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7