Commit 8c0166b9 authored by GitLab Bot's avatar GitLab Bot

Add latest changes from gitlab-org/gitlab@master

parent 133924c6
...@@ -34,8 +34,8 @@ recover. See below for more details. ...@@ -34,8 +34,8 @@ recover. See below for more details.
The following guide assumes that: The following guide assumes that:
- You are using Omnibus and therefore you are using PostgreSQL 9.6 or later - You are using Omnibus and therefore you are using PostgreSQL 9.6 or later
which includes the [`pg_basebackup` tool][pgback] and improved which includes the [`pg_basebackup` tool](https://www.postgresql.org/docs/9.6/app-pgbasebackup.html) and improved
[Foreign Data Wrapper][FDW] support. [Foreign Data Wrapper][FDW](https://www.postgresql.org/docs/9.6/postgres-fdw.html) support.
- You have a **primary** node already set up (the GitLab server you are - You have a **primary** node already set up (the GitLab server you are
replicating from), running Omnibus' PostgreSQL (or equivalent version), and replicating from), running Omnibus' PostgreSQL (or equivalent version), and
you have a new **secondary** server set up with the same versions of the OS, you have a new **secondary** server set up with the same versions of the OS,
...@@ -146,9 +146,9 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o ...@@ -146,9 +146,9 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
address (corresponds to "internal address" for Google Cloud Platform) for address (corresponds to "internal address" for Google Cloud Platform) for
`postgresql['md5_auth_cidr_addresses']` and `postgresql['listen_address']`. `postgresql['md5_auth_cidr_addresses']` and `postgresql['listen_address']`.
The `listen_address` option opens PostgreSQL up to network connections The `listen_address` option opens PostgreSQL up to network connections with the interface
with the interface corresponding to the given address. See [the PostgreSQL corresponding to the given address. See [the PostgreSQL documentation](https://www.postgresql.org/docs/9.6/runtime-config-connection.html)
documentation][pg-docs-runtime-conn] for more details. for more details.
Depending on your network configuration, the suggested addresses may not Depending on your network configuration, the suggested addresses may not
be correct. If your **primary** node and **secondary** nodes connect over a local be correct. If your **primary** node and **secondary** nodes connect over a local
...@@ -199,9 +199,8 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o ...@@ -199,9 +199,8 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o
postgresql['md5_auth_cidr_addresses'] = ['<primary_node_ip>/32', '<secondary_node_ip>/32', '<another_secondary_node_ip>/32'] postgresql['md5_auth_cidr_addresses'] = ['<primary_node_ip>/32', '<secondary_node_ip>/32', '<another_secondary_node_ip>/32']
``` ```
You may also want to edit the `wal_keep_segments` and `max_wal_senders` to You may also want to edit the `wal_keep_segments` and `max_wal_senders` to match your
match your database replication requirements. Consult the [PostgreSQL - database replication requirements. Consult the [PostgreSQL - Replication documentation](https://www.postgresql.org/docs/9.6/runtime-config-replication.html)
Replication documentation][pg-docs-runtime-replication]
for more information. for more information.
1. Save the file and reconfigure GitLab for the database listen changes and 1. Save the file and reconfigure GitLab for the database listen changes and
...@@ -427,7 +426,7 @@ data before running `pg_basebackup`. ...@@ -427,7 +426,7 @@ data before running `pg_basebackup`.
(e.g., you know the network path is secure, or you are using a site-to-site (e.g., you know the network path is secure, or you are using a site-to-site
VPN). This is **not** safe over the public Internet! VPN). This is **not** safe over the public Internet!
- You can read more details about each `sslmode` in the - You can read more details about each `sslmode` in the
[PostgreSQL documentation][pg-docs-ssl]; [PostgreSQL documentation](https://www.postgresql.org/docs/9.6/libpq-ssl.html#LIBPQ-SSL-PROTECTION);
the instructions above are carefully written to ensure protection against the instructions above are carefully written to ensure protection against
both passive eavesdroppers and active "man-in-the-middle" attackers. both passive eavesdroppers and active "man-in-the-middle" attackers.
- Change the `--slot-name` to the name of the replication slot - Change the `--slot-name` to the name of the replication slot
...@@ -449,7 +448,7 @@ high-availability configuration with a cluster of nodes supporting a Geo ...@@ -449,7 +448,7 @@ high-availability configuration with a cluster of nodes supporting a Geo
information, see [High Availability with GitLab Omnibus](../../high_availability/database.md#high-availability-with-gitlab-omnibus-premium-only). information, see [High Availability with GitLab Omnibus](../../high_availability/database.md#high-availability-with-gitlab-omnibus-premium-only).
For a Geo **secondary** node to work properly with PgBouncer in front of the database, For a Geo **secondary** node to work properly with PgBouncer in front of the database,
it will need a separate read-only user to make [PostgreSQL FDW queries][FDW] it will need a separate read-only user to make [PostgreSQL FDW queries](https://www.postgresql.org/docs/9.6/postgres-fdw.html)
work: work:
1. On the **primary** Geo database, enter the PostgreSQL on the console as an 1. On the **primary** Geo database, enter the PostgreSQL on the console as an
...@@ -495,11 +494,6 @@ work: ...@@ -495,11 +494,6 @@ work:
Read the [troubleshooting document](troubleshooting.md). Read the [troubleshooting document](troubleshooting.md).
[replication-slots-article]: https://medium.com/@tk512/replication-slots-in-postgresql-b4b03d277c75 [replication-slots-article]: https://medium.com/@tk512/replication-slots-in-postgresql-b4b03d277c75
[pgback]: http://www.postgresql.org/docs/9.2/static/app-pgbasebackup.html
[replication user]:https://wiki.postgresql.org/wiki/Streaming_Replication [replication user]:https://wiki.postgresql.org/wiki/Streaming_Replication
[FDW]: https://www.postgresql.org/docs/9.6/static/postgres-fdw.html
[toc]: index.md#using-omnibus-gitlab [toc]: index.md#using-omnibus-gitlab
[rake-maintenance]: ../../raketasks/maintenance.md [rake-maintenance]: ../../raketasks/maintenance.md
[pg-docs-ssl]: https://www.postgresql.org/docs/9.6/static/libpq-ssl.html#LIBPQ-SSL-PROTECTION
[pg-docs-runtime-conn]: https://www.postgresql.org/docs/9.6/static/runtime-config-connection.html
[pg-docs-runtime-replication]: https://www.postgresql.org/docs/9.6/static/runtime-config-replication.html
...@@ -132,7 +132,7 @@ when `roles ['geo_secondary_role']` is set. For high availability, ...@@ -132,7 +132,7 @@ when `roles ['geo_secondary_role']` is set. For high availability,
refer to [Geo High Availability](../../high_availability/README.md). refer to [Geo High Availability](../../high_availability/README.md).
If you want to run this database external to Omnibus, please follow the instructions below. If you want to run this database external to Omnibus, please follow the instructions below.
The tracking database requires an [FDW](https://www.postgresql.org/docs/9.6/static/postgres-fdw.html) The tracking database requires an [FDW](https://www.postgresql.org/docs/9.6/postgres-fdw.html)
connection with the **secondary** replica database for improved performance. connection with the **secondary** replica database for improved performance.
If you have an external database ready to be used as the tracking database, If you have an external database ready to be used as the tracking database,
...@@ -173,7 +173,7 @@ the tracking database on port 5432. ...@@ -173,7 +173,7 @@ the tracking database on port 5432.
gitlab-rake geo:db:migrate gitlab-rake geo:db:migrate
``` ```
1. Configure the [PostgreSQL FDW](https://www.postgresql.org/docs/9.6/static/postgres-fdw.html) 1. Configure the [PostgreSQL FDW](https://www.postgresql.org/docs/9.6/postgres-fdw.html)
connection and credentials: connection and credentials:
Save the script below in a file, ex. `/tmp/geo_fdw.sh` and modify the connection Save the script below in a file, ex. `/tmp/geo_fdw.sh` and modify the connection
......
...@@ -108,7 +108,7 @@ The following are required to run Geo: ...@@ -108,7 +108,7 @@ The following are required to run Geo:
[fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md)) [fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md))
The following operating systems are known to ship with a current version of OpenSSH: The following operating systems are known to ship with a current version of OpenSSH:
- [CentOS](https://www.centos.org) 7.4+ - [CentOS](https://www.centos.org) 7.4+
- [Ubuntu](https://www.ubuntu.com) 16.04+ - [Ubuntu](https://ubuntu.com) 16.04+
- PostgreSQL 9.6+ with [FDW](https://www.postgresql.org/docs/9.6/postgres-fdw.html) support and [Streaming Replication](https://wiki.postgresql.org/wiki/Streaming_Replication) - PostgreSQL 9.6+ with [FDW](https://www.postgresql.org/docs/9.6/postgres-fdw.html) support and [Streaming Replication](https://wiki.postgresql.org/wiki/Streaming_Replication)
- Git 2.9+ - Git 2.9+
- All nodes must run the same GitLab version. - All nodes must run the same GitLab version.
......
# Geo security review (Q&A) **(PREMIUM ONLY)** # Geo security review (Q&A) **(PREMIUM ONLY)**
The following security review of the Geo feature set focuses on security The following security review of the Geo feature set focuses on security aspects of
aspects of the feature as they apply to customers running their own GitLab the feature as they apply to customers running their own GitLab instances. The review
instances. The review questions are based in part on the [application security architecture](https://www.owasp.org/index.php/Application_Security_Architecture_Cheat_Sheet) questions are based in part on the [OWASP Application Security Verification Standard Project](https://www.owasp.org/index.php/Category:OWASP_Application_Security_Verification_Standard_Project)
questions from [owasp.org](https://www.owasp.org). from [owasp.org](https://www.owasp.org/index.php/Main_Page).
## Business Model ## Business Model
...@@ -30,7 +30,7 @@ questions from [owasp.org](https://www.owasp.org). ...@@ -30,7 +30,7 @@ questions from [owasp.org](https://www.owasp.org).
private projects. Geo replicates them all indiscriminately. “Selective sync” private projects. Geo replicates them all indiscriminately. “Selective sync”
exists for files and repositories (but not database content), which would permit exists for files and repositories (but not database content), which would permit
only less-sensitive projects to be replicated to a **secondary** node if desired. only less-sensitive projects to be replicated to a **secondary** node if desired.
- See also: [developing a data classification policy](https://gitlab.com/gitlab-com/security/issues/4). - See also: [GitLab data classification policy](https://about.gitlab.com/handbook/engineering/security/data-classification-policy.html).
### What data backup and retention requirements have been defined for the application? ### What data backup and retention requirements have been defined for the application?
......
...@@ -174,4 +174,4 @@ After this, the cluster should start back up, and the server agents rejoin. Shor ...@@ -174,4 +174,4 @@ After this, the cluster should start back up, and the server agents rejoin. Shor
#### Recover a failed cluster #### Recover a failed cluster
If you have taken advantage of Consul to store other data, and want to restore the failed cluster, please follow the [Consul guide](https://www.consul.io/docs/guides/outage.html) to recover a failed cluster. If you have taken advantage of Consul to store other data, and want to restore the failed cluster, please follow the [Consul guide](https://learn.hashicorp.com/consul/day-2-operations/outage) to recover a failed cluster.
...@@ -926,7 +926,7 @@ repmgr['trust_auth_cidr_addresses'] = %w(192.168.1.44/32 db2.example.com) ...@@ -926,7 +926,7 @@ repmgr['trust_auth_cidr_addresses'] = %w(192.168.1.44/32 db2.example.com)
##### MD5 Authentication ##### MD5 Authentication
If you are running on an untrusted network, repmgr can use md5 authentication If you are running on an untrusted network, repmgr can use md5 authentication
with a [.pgpass file](https://www.postgresql.org/docs/9.6/static/libpq-pgpass.html) with a [.pgpass file](https://www.postgresql.org/docs/9.6/libpq-pgpass.html)
to authenticate. to authenticate.
You can specify by IP address, FQDN, or by subnet, using the same format as in You can specify by IP address, FQDN, or by subnet, using the same format as in
......
...@@ -40,7 +40,7 @@ these additional steps before proceeding with GitLab installation. ...@@ -40,7 +40,7 @@ these additional steps before proceeding with GitLab installation.
``` ```
1. Download/install GitLab Omnibus using **steps 1 and 2** from 1. Download/install GitLab Omnibus using **steps 1 and 2** from
[GitLab downloads](https://about.gitlab.com/downloads). Do not complete other [GitLab downloads](https://about.gitlab.com/install/). Do not complete other
steps on the download page. steps on the download page.
1. Create/edit `/etc/gitlab/gitlab.rb` and use the following configuration. 1. Create/edit `/etc/gitlab/gitlab.rb` and use the following configuration.
Be sure to change the `external_url` to match your eventual GitLab front-end Be sure to change the `external_url` to match your eventual GitLab front-end
......
...@@ -117,7 +117,7 @@ across NFS. The GitLab support team will not be able to assist on performance is ...@@ -117,7 +117,7 @@ across NFS. The GitLab support team will not be able to assist on performance is
this configuration. this configuration.
Additionally, this configuration is specifically warned against in the Additionally, this configuration is specifically warned against in the
[Postgres Documentation](https://www.postgresql.org/docs/current/static/creating-cluster.html#CREATING-CLUSTER-NFS): [Postgres Documentation](https://www.postgresql.org/docs/current/creating-cluster.html#CREATING-CLUSTER-NFS):
>PostgreSQL does nothing special for NFS file systems, meaning it assumes NFS behaves exactly like >PostgreSQL does nothing special for NFS file systems, meaning it assumes NFS behaves exactly like
>locally-connected drives. If the client or server NFS implementation does not provide standard file >locally-connected drives. If the client or server NFS implementation does not provide standard file
......
...@@ -11,7 +11,7 @@ setup act as clients while the NFS server plays host. ...@@ -11,7 +11,7 @@ setup act as clients while the NFS server plays host.
> Note: The instructions provided in this documentation allow for setting a quick > Note: The instructions provided in this documentation allow for setting a quick
proof of concept but will leave NFS as potential single point of failure and proof of concept but will leave NFS as potential single point of failure and
therefore not recommended for use in production. Explore options such as [Pacemaker therefore not recommended for use in production. Explore options such as [Pacemaker
and Corosync](http://clusterlabs.org/) for highly available NFS in production. and Corosync](https://clusterlabs.org) for highly available NFS in production.
Below are instructions for setting up an application node(client) in an HA cluster Below are instructions for setting up an application node(client) in an HA cluster
to read from and write to a central NFS server(host). to read from and write to a central NFS server(host).
......
...@@ -313,12 +313,12 @@ Pick the one that suits your needs. ...@@ -313,12 +313,12 @@ Pick the one that suits your needs.
- [Installations from source][source]: You need to install Redis and Sentinel - [Installations from source][source]: You need to install Redis and Sentinel
yourself. Use the [Redis HA installation from source](redis_source.md) yourself. Use the [Redis HA installation from source](redis_source.md)
documentation. documentation.
- [Omnibus GitLab **Community Edition** (CE) package][ce]: Redis is bundled, so you - [Omnibus GitLab **Community Edition** (CE) package](https://about.gitlab.com/install/?version=ce): Redis is bundled, so you
can use the package with only the Redis service enabled as described in steps can use the package with only the Redis service enabled as described in steps
1 and 2 of this document (works for both master and slave setups). To install 1 and 2 of this document (works for both master and slave setups). To install
and configure Sentinel, jump directly to the Sentinel section in the and configure Sentinel, jump directly to the Sentinel section in the
[Redis HA installation from source](redis_source.md#step-3-configuring-the-redis-sentinel-instances) documentation. [Redis HA installation from source](redis_source.md#step-3-configuring-the-redis-sentinel-instances) documentation.
- [Omnibus GitLab **Enterprise Edition** (EE) package][ee]: Both Redis and Sentinel - [Omnibus GitLab **Enterprise Edition** (EE) package](https://about.gitlab.com/install/?version=ee): Both Redis and Sentinel
are bundled in the package, so you can use the EE package to set up the whole are bundled in the package, so you can use the EE package to set up the whole
Redis HA infrastructure (master, slave and Sentinel) which is described in Redis HA infrastructure (master, slave and Sentinel) which is described in
this document. this document.
...@@ -1025,6 +1025,4 @@ Read more on High Availability: ...@@ -1025,6 +1025,4 @@ Read more on High Availability:
[sentinel]: https://redis.io/topics/sentinel [sentinel]: https://redis.io/topics/sentinel
[omnifile]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/libraries/gitlab_rails.rb [omnifile]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/libraries/gitlab_rails.rb
[source]: ../../install/installation.md [source]: ../../install/installation.md
[ce]: https://about.gitlab.com/downloads
[ee]: https://about.gitlab.com/downloads-ee
[it]: https://gitlab.com/gitlab-org/gitlab-foss/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png [it]: https://gitlab.com/gitlab-org/gitlab-foss/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png
...@@ -12,10 +12,12 @@ information on CI/CD pipeline configuration in GitLab, see the ...@@ -12,10 +12,12 @@ information on CI/CD pipeline configuration in GitLab, see the
Get all GitLab CI YML templates. Get all GitLab CI YML templates.
``` ```plaintext
GET /templates/gitlab_ci_ymls GET /templates/gitlab_ci_ymls
``` ```
Example request:
```bash ```bash
curl https://gitlab.example.com/api/v4/templates/gitlab_ci_ymls curl https://gitlab.example.com/api/v4/templates/gitlab_ci_ymls
``` ```
...@@ -28,6 +30,10 @@ Example response: ...@@ -28,6 +30,10 @@ Example response:
"key": "Android", "key": "Android",
"name": "Android" "name": "Android"
}, },
{
"key": "Android-Fastlane",
"name": "Android-Fastlane"
},
{ {
"key": "Auto-DevOps", "key": "Auto-DevOps",
"name": "Auto-DevOps" "name": "Auto-DevOps"
...@@ -48,6 +54,10 @@ Example response: ...@@ -48,6 +54,10 @@ Example response:
"key": "Clojure", "key": "Clojure",
"name": "Clojure" "name": "Clojure"
}, },
{
"key": "Code-Quality",
"name": "Code-Quality"
},
{ {
"key": "Crystal", "key": "Crystal",
"name": "Crystal" "name": "Crystal"
...@@ -95,14 +105,6 @@ Example response: ...@@ -95,14 +105,6 @@ Example response:
{ {
"key": "Mono", "key": "Mono",
"name": "Mono" "name": "Mono"
},
{
"key": "Nodejs",
"name": "Nodejs"
},
{
"key": "OpenShift",
"name": "OpenShift"
} }
] ]
``` ```
...@@ -111,14 +113,16 @@ Example response: ...@@ -111,14 +113,16 @@ Example response:
Get a single GitLab CI YML template. Get a single GitLab CI YML template.
``` ```plaintext
GET /templates/gitlab_ci_ymls/:key GET /templates/gitlab_ci_ymls/:key
``` ```
| Attribute | Type | Required | Description | | Attribute | Type | Required | Description |
| ---------- | ------ | -------- | ----------- | | ---------- | ------ | -------- | ------------------------------------- |
| `key` | string | yes | The key of the GitLab CI YML template | | `key` | string | yes | The key of the GitLab CI YML template |
Example request:
```bash ```bash
curl https://gitlab.example.com/api/v4/templates/gitlab_ci_ymls/Ruby curl https://gitlab.example.com/api/v4/templates/gitlab_ci_ymls/Ruby
``` ```
......
...@@ -155,7 +155,8 @@ docker-in-docker service and ...@@ -155,7 +155,8 @@ docker-in-docker service and
[GitLab.com Shared Runners](../../user/gitlab_com/index.html#shared-runners) [GitLab.com Shared Runners](../../user/gitlab_com/index.html#shared-runners)
support this. support this.
1. Install [GitLab Runner](https://docs.gitlab.com/runner/install). 1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/).
1. Register GitLab Runner from the command line to use `docker` and `privileged` 1. Register GitLab Runner from the command line to use `docker` and `privileged`
mode: mode:
...@@ -315,7 +316,8 @@ are done to the services as well, making these incompatible. ...@@ -315,7 +316,8 @@ are done to the services as well, making these incompatible.
In order to do that, follow the steps: In order to do that, follow the steps:
1. Install [GitLab Runner](https://docs.gitlab.com/runner/install). 1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/).
1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: 1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`:
```bash ```bash
......
...@@ -22,7 +22,7 @@ and the basics of game development. ...@@ -22,7 +22,7 @@ and the basics of game development.
Our [demo game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/) consists of a simple spaceship traveling in space that shoots by clicking the mouse in a given direction. Our [demo game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/) consists of a simple spaceship traveling in space that shoots by clicking the mouse in a given direction.
Creating a strong CI/CD pipeline at the beginning of developing another game, [Dark Nova](http://darknova.io/), Creating a strong CI/CD pipeline at the beginning of developing another game, [Dark Nova](https://www.darknova.io),
was essential for the fast pace the team worked at. This tutorial will build upon my was essential for the fast pace the team worked at. This tutorial will build upon my
[previous introductory article](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) and go through the following steps: [previous introductory article](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) and go through the following steps:
...@@ -509,7 +509,7 @@ deploy: ...@@ -509,7 +509,7 @@ deploy:
Within the [demo repository](https://gitlab.com/blitzgren/gitlab-game-demo) you can also find a handful of boilerplate code to get Within the [demo repository](https://gitlab.com/blitzgren/gitlab-game-demo) you can also find a handful of boilerplate code to get
[Typescript](https://www.typescriptlang.org/), [Mocha](https://mochajs.org/), [Gulp](https://gulpjs.com/) and [Phaser](https://phaser.io) all playing [Typescript](https://www.typescriptlang.org/), [Mocha](https://mochajs.org/), [Gulp](https://gulpjs.com/) and [Phaser](https://phaser.io) all playing
together nicely with GitLab CI/CD, which is the result of lessons learned while making [Dark Nova](http://darknova.io/). together nicely with GitLab CI/CD, which is the result of lessons learned while making [Dark Nova](https://www.darknova.io).
Using a combination of free and open source software, we have a full CI/CD pipeline, a game foundation, Using a combination of free and open source software, we have a full CI/CD pipeline, a game foundation,
and unit tests, all running and deployed at every push to master - with shockingly little code. and unit tests, all running and deployed at every push to master - with shockingly little code.
Errors can be easily debugged through GitLab's build logs, and within minutes of a successful commit, Errors can be easily debugged through GitLab's build logs, and within minutes of a successful commit,
...@@ -527,6 +527,6 @@ Here are some ideas to further investigate that can speed up or improve your pip ...@@ -527,6 +527,6 @@ Here are some ideas to further investigate that can speed up or improve your pip
- [Yarn](https://yarnpkg.com) instead of npm - [Yarn](https://yarnpkg.com) instead of npm
- Set up a custom [Docker](../../../ci/docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) image that can preload dependencies and tools (like AWS CLI) - Set up a custom [Docker](../../../ci/docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) image that can preload dependencies and tools (like AWS CLI)
- Forward a [custom domain](https:/docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) to your game's S3 static website - Forward a [custom domain](https://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) to your game's S3 static website
- Combine jobs if you find it unnecessary for a small project - Combine jobs if you find it unnecessary for a small project
- Avoid the queues and set up your own [custom GitLab CI/CD runner](https://about.gitlab.com/2016/03/01/gitlab-runner-with-docker/) - Avoid the queues and set up your own [custom GitLab CI/CD runner](https://about.gitlab.com/2016/03/01/gitlab-runner-with-docker/)
...@@ -47,14 +47,14 @@ infrastructure is up and running, and that your units of code work well together ...@@ -47,14 +47,14 @@ infrastructure is up and running, and that your units of code work well together
[Selenium](http://www.seleniumhq.org/) is a piece of software that can control web browsers, e.g., to make them [Selenium](http://www.seleniumhq.org/) is a piece of software that can control web browsers, e.g., to make them
visit a specific URL or interact with elements on the page. It can be programmatically controlled visit a specific URL or interact with elements on the page. It can be programmatically controlled
from a variety of programming languages. In this article we're going to be using the from a variety of programming languages. In this article we're going to be using the
[WebdriverIO](http://webdriver.io/) JavaScript bindings, but the general concept should carry over [WebdriverIO](https://webdriver.io/) JavaScript bindings, but the general concept should carry over
pretty well to pretty well to
[other programming languages supported by Selenium](http://docs.seleniumhq.org/about/platforms.jsp#programming-languages). [other programming languages supported by Selenium](http://docs.seleniumhq.org/about/platforms.jsp#programming-languages).
## Writing tests ## Writing tests
You can write tests using You can write tests using
[several testing frameworks supported by WebdriverIO](http://webdriver.io/guide/testrunner/frameworks.html). [several testing frameworks supported by WebdriverIO](https://webdriver.io/guide/testrunner/frameworks.html).
We will be using [Jasmine](https://jasmine.github.io/) here: We will be using [Jasmine](https://jasmine.github.io/) here:
```javascript ```javascript
...@@ -79,14 +79,14 @@ multiple tests, such as making sure you are logged in. ...@@ -79,14 +79,14 @@ multiple tests, such as making sure you are logged in.
The function `it` defines an individual test. The function `it` defines an individual test.
[The `browser` object](http://webdriver.io/guide/testrunner/browserobject.html) is WebdriverIO's [The `browser` object](https://webdriver.io/guide/testrunner/browserobject.html) is WebdriverIO's
special sauce. It provides most of [the WebdriverIO API methods](http://webdriver.io/api.html) that are the key to special sauce. It provides most of [the WebdriverIO API methods](https://webdriver.io/api.html) that are the key to
steering the browser. In this case, we can use steering the browser. In this case, we can use
[`browser.url`](http://webdriver.io/api/protocol/url.html) to visit `/page-that-does-not-exist` to [`browser.url`](https://webdriver.io/api/protocol/url.html) to visit `/page-that-does-not-exist` to
hit our 404 page. We can then use [`browser.getUrl`](http://webdriver.io/api/property/getUrl.html) hit our 404 page. We can then use [`browser.getUrl`](https://webdriver.io/api/property/getUrl.html)
to verify that the current page is indeed at the location we specified. To interact with the page, to verify that the current page is indeed at the location we specified. To interact with the page,
we can simply pass CSS selectors to we can simply pass CSS selectors to
[`browser.element`](http://webdriver.io/api/protocol/element.html) to get access to elements on the [`browser.element`](https://webdriver.io/api/protocol/element.html) to get access to elements on the
page and to interact with them - for example, to click on the link back to the home page. page and to interact with them - for example, to click on the link back to the home page.
The simple test shown above The simple test shown above
...@@ -107,9 +107,9 @@ you can use [the Webpack Dev Server WebdriverIO plugin](https://www.npmjs.com/pa ...@@ -107,9 +107,9 @@ you can use [the Webpack Dev Server WebdriverIO plugin](https://www.npmjs.com/pa
that automatically starts a development server before executing the tests. that automatically starts a development server before executing the tests.
The WebdriverIO documentation has The WebdriverIO documentation has
[an overview of all configuration options](http://webdriver.io/guide/getstarted/configuration.html), but the [an overview of all configuration options](https://webdriver.io/guide/getstarted/configuration.html), but the
easiest way to get started is to start with easiest way to get started is to start with
[WebdriverIO's default configuration](http://webdriver.io/guide/testrunner/configurationfile.html), which [WebdriverIO's default configuration](https://webdriver.io/guide/testrunner/configurationfile.html), which
provides an overview of all available options. The two options that are going to be most relevant now are the provides an overview of all available options. The two options that are going to be most relevant now are the
`specs` option, which is an array of paths to your tests, and the `baseUrl` option, which points to where your app is `specs` option, which is an array of paths to your tests, and the `baseUrl` option, which points to where your app is
running. And finally, we will need to tell WebdriverIO in which browsers we would like to run our running. And finally, we will need to tell WebdriverIO in which browsers we would like to run our
...@@ -182,7 +182,7 @@ e2e:chrome: ...@@ -182,7 +182,7 @@ e2e:chrome:
Now that we have a job to run the end-to-end tests in, we need to tell WebdriverIO how to connect to Now that we have a job to run the end-to-end tests in, we need to tell WebdriverIO how to connect to
the Selenium servers running alongside it. We've already cheated a bit above by the Selenium servers running alongside it. We've already cheated a bit above by
passing the value of the [`host`](http://webdriver.io/guide/getstarted/configuration.html#host) passing the value of the [`host`](https://webdriver.io/guide/getstarted/configuration.html#host)
option as an argument to `npm run confidence-check` on the command line. option as an argument to `npm run confidence-check` on the command line.
However, we still need to tell WebdriverIO which browser is available for it to use. However, we still need to tell WebdriverIO which browser is available for it to use.
...@@ -248,7 +248,7 @@ production project, see: ...@@ -248,7 +248,7 @@ production project, see:
- [Flockademic's `.gitlab-ci.yml`](https://gitlab.com/Flockademic/Flockademic/blob/dev/.gitlab-ci.yml) - [Flockademic's `.gitlab-ci.yml`](https://gitlab.com/Flockademic/Flockademic/blob/dev/.gitlab-ci.yml)
- [Flockademic's tests](https://gitlab.com/Flockademic/Flockademic/tree/dev/__e2e__) - [Flockademic's tests](https://gitlab.com/Flockademic/Flockademic/tree/dev/__e2e__)
There's plenty more that WebdriverIO can do. For example, you can configure a [`screenshotPath`](http://webdriver.io/guide/getstarted/configuration.html#screenshotPath) to tell WebdriverIO to take There's plenty more that WebdriverIO can do. For example, you can configure a [`screenshotPath`](https://webdriver.io/guide/getstarted/configuration.html#screenshotPath) to tell WebdriverIO to take
a screenshot when tests are failing. Then tell GitLab CI/CD to store those a screenshot when tests are failing. Then tell GitLab CI/CD to store those
[artifacts](../../yaml/README.md#artifacts), and you'll be able to see what went [artifacts](../../yaml/README.md#artifacts), and you'll be able to see what went
wrong within GitLab. wrong within GitLab.
...@@ -15,7 +15,7 @@ last_updated: 2019-03-06 ...@@ -15,7 +15,7 @@ last_updated: 2019-03-06
GitLab features our applications with Continuous Integration, and it is possible to easily deploy the new code changes to the production server whenever we want. GitLab features our applications with Continuous Integration, and it is possible to easily deploy the new code changes to the production server whenever we want.
In this tutorial, we'll show you how to initialize a [Laravel](http://laravel.com/) application and set up our [Envoy](https://laravel.com/docs/envoy) tasks, then we'll jump into see how to test and deploy it with [GitLab CI/CD](../README.md) via [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/). In this tutorial, we'll show you how to initialize a [Laravel](https://laravel.com) application and set up our [Envoy](https://laravel.com/docs/master/envoy) tasks, then we'll jump into see how to test and deploy it with [GitLab CI/CD](../README.md) via [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/).
We assume you have a basic experience with Laravel, Linux servers, We assume you have a basic experience with Laravel, Linux servers,
and you know how to use GitLab. and you know how to use GitLab.
...@@ -25,11 +25,11 @@ It has a great community with a [fantastic documentation](https://laravel.com/do ...@@ -25,11 +25,11 @@ It has a great community with a [fantastic documentation](https://laravel.com/do
Aside from the usual routing, controllers, requests, responses, views, and (blade) templates, out of the box Laravel provides plenty of additional services such as cache, events, localization, authentication and many others. Aside from the usual routing, controllers, requests, responses, views, and (blade) templates, out of the box Laravel provides plenty of additional services such as cache, events, localization, authentication and many others.
We will use [Envoy](https://laravel.com/docs/master/envoy) as an SSH task runner based on PHP. We will use [Envoy](https://laravel.com/docs/master/envoy) as an SSH task runner based on PHP.
It uses a clean, minimal [Blade syntax](https://laravel.com/docs/blade) to set up tasks that can run on remote servers, such as, cloning your project from the repository, installing the Composer dependencies, and running [Artisan commands](https://laravel.com/docs/artisan). It uses a clean, minimal [Blade syntax](https://laravel.com/docs/master/blade) to set up tasks that can run on remote servers, such as, cloning your project from the repository, installing the Composer dependencies, and running [Artisan commands](https://laravel.com/docs/master/artisan).
## Initialize our Laravel app on GitLab ## Initialize our Laravel app on GitLab
We assume [you have installed a new laravel project](https://laravel.com/docs/installation#installation), so let's start with a unit test, and initialize Git for the project. We assume [you have installed a new laravel project](https://laravel.com/docs/master/installation#installation), so let's start with a unit test, and initialize Git for the project.
### Unit Test ### Unit Test
...@@ -87,7 +87,7 @@ We have installed LEMP stack which stands for Linux, NGINX, MySQL and PHP on our ...@@ -87,7 +87,7 @@ We have installed LEMP stack which stands for Linux, NGINX, MySQL and PHP on our
### Create a new user ### Create a new user
Let's now create a new user that will be used to deploy our website and give it Let's now create a new user that will be used to deploy our website and give it
the needed permissions using [Linux ACL](https://serversforhackers.com/video/linux-acls): the needed permissions using [Linux ACL](https://serversforhackers.com/c/linux-acls):
```bash ```bash
# Create user deployer # Create user deployer
...@@ -179,7 +179,7 @@ You may replace the app's name in `/var/www/app/current/public` with the folder ...@@ -179,7 +179,7 @@ You may replace the app's name in `/var/www/app/current/public` with the folder
So we have our Laravel app ready for production. So we have our Laravel app ready for production.
The next thing is to use Envoy to perform the deploy. The next thing is to use Envoy to perform the deploy.
To use Envoy, we should first install it on our local machine [using the given instructions by Laravel](https://laravel.com/docs/envoy/#introduction). To use Envoy, we should first install it on our local machine [using the given instructions by Laravel](https://laravel.com/docs/master/envoy/#introduction).
### How Envoy works ### How Envoy works
...@@ -216,7 +216,7 @@ Our deployment plan is to clone the latest release from GitLab repository, insta ...@@ -216,7 +216,7 @@ Our deployment plan is to clone the latest release from GitLab repository, insta
#### @setup directive #### @setup directive
The first step of our deployment process is to define a set of variables within [@setup](https://laravel.com/docs/envoy/#setup) directive. The first step of our deployment process is to define a set of variables within [@setup](https://laravel.com/docs/master/envoy/#setup) directive.
You may change the `app` to your application's name: You may change the `app` to your application's name:
```php ```php
...@@ -241,7 +241,7 @@ You may change the `app` to your application's name: ...@@ -241,7 +241,7 @@ You may change the `app` to your application's name:
#### @story directive #### @story directive
The [@story](https://laravel.com/docs/envoy/#stories) directive allows us define a list of tasks that can be run as a single task. The [@story](https://laravel.com/docs/master/envoy/#stories) directive allows us define a list of tasks that can be run as a single task.
Here we have three tasks called `clone_repository`, `run_composer`, `update_symlinks`. These variables are usable to making our task's codes more cleaner: Here we have three tasks called `clone_repository`, `run_composer`, `update_symlinks`. These variables are usable to making our task's codes more cleaner:
```php ```php
...@@ -394,7 +394,7 @@ We have our app ready on GitLab, and we also can deploy it manually. ...@@ -394,7 +394,7 @@ We have our app ready on GitLab, and we also can deploy it manually.
But let's take a step forward to do it automatically with [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) method. But let's take a step forward to do it automatically with [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) method.
We need to check every commit with a set of automated tests to become aware of issues at the earliest, and then, we can deploy to the target environment if we are happy with the result of the tests. We need to check every commit with a set of automated tests to become aware of issues at the earliest, and then, we can deploy to the target environment if we are happy with the result of the tests.
[GitLab CI/CD](../../README.md) allows us to use [Docker](https://docker.com/) engine to handle the process of testing and deploying our app. [GitLab CI/CD](../../README.md) allows us to use [Docker](https://www.docker.com) engine to handle the process of testing and deploying our app.
In the case you're not familiar with Docker, refer to [How to Automate Docker Deployments](http://paislee.io/how-to-automate-docker-deployments/). In the case you're not familiar with Docker, refer to [How to Automate Docker Deployments](http://paislee.io/how-to-automate-docker-deployments/).
To be able to build, test, and deploy our app with GitLab CI/CD, we need to prepare our work environment. To be able to build, test, and deploy our app with GitLab CI/CD, we need to prepare our work environment.
...@@ -431,7 +431,7 @@ RUN curl --silent --show-error https://getcomposer.org/installer | php -- --inst ...@@ -431,7 +431,7 @@ RUN curl --silent --show-error https://getcomposer.org/installer | php -- --inst
RUN composer global require "laravel/envoy=~1.0" RUN composer global require "laravel/envoy=~1.0"
``` ```
We added the [official PHP 7.1 Docker image](https://hub.docker.com/r/_/php/), which consist of a minimum installation of Debian Jessie with PHP pre-installed, and works perfectly for our use case. We added the [official PHP 7.1 Docker image](https://hub.docker.com/_/php), which consist of a minimum installation of Debian Jessie with PHP pre-installed, and works perfectly for our use case.
We used `docker-php-ext-install` (provided by the official PHP Docker image) to install the PHP extensions we need. We used `docker-php-ext-install` (provided by the official PHP Docker image) to install the PHP extensions we need.
...@@ -557,7 +557,7 @@ GitLab CI/CD allows us to use [environment variables](../../yaml/README.md#varia ...@@ -557,7 +557,7 @@ GitLab CI/CD allows us to use [environment variables](../../yaml/README.md#varia
We defined MySQL as our database management system, which comes with a superuser root created by default. We defined MySQL as our database management system, which comes with a superuser root created by default.
So we should adjust the configuration of MySQL instance by defining `MYSQL_DATABASE` variable as our database name and `MYSQL_ROOT_PASSWORD` variable as the password of `root`. So we should adjust the configuration of MySQL instance by defining `MYSQL_DATABASE` variable as our database name and `MYSQL_ROOT_PASSWORD` variable as the password of `root`.
Find out more about MySQL variables at the [official MySQL Docker Image](https://hub.docker.com/r/_/mysql/). Find out more about MySQL variables at the [official MySQL Docker Image](https://hub.docker.com/_/mysql).
Also set the variables `DB_HOST` to `mysql` and `DB_USERNAME` to `root`, which are Laravel specific variables. Also set the variables `DB_HOST` to `mysql` and `DB_USERNAME` to `root`, which are Laravel specific variables.
We define `DB_HOST` as `mysql` instead of `127.0.0.1`, as we use MySQL Docker image as a service which [is linked to the main Docker image](../../docker/using_docker_images.md#how-services-are-linked-to-the-job). We define `DB_HOST` as `mysql` instead of `127.0.0.1`, as we use MySQL Docker image as a service which [is linked to the main Docker image](../../docker/using_docker_images.md#how-services-are-linked-to-the-job).
...@@ -634,7 +634,7 @@ deploy_production: ...@@ -634,7 +634,7 @@ deploy_production:
- master - master
``` ```
You may also want to add another job for [staging environment](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments), to final test your application before deploying to production. You may also want to add another job for [staging environment](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/), to final test your application before deploying to production.
### Turn on GitLab CI/CD ### Turn on GitLab CI/CD
......
...@@ -72,7 +72,7 @@ You can do this through the [Dashboard](https://dashboard.heroku.com/). ...@@ -72,7 +72,7 @@ You can do this through the [Dashboard](https://dashboard.heroku.com/).
First install [Docker Engine](https://docs.docker.com/installation/). First install [Docker Engine](https://docs.docker.com/installation/).
To build this project you also need to have [GitLab Runner](https://docs.gitlab.com/runner). To build this project you also need to have [GitLab Runner](https://docs.gitlab.com/runner/index.html).
You can use public runners available on `gitlab.com` or you can register your own: You can use public runners available on `gitlab.com` or you can register your own:
```sh ```sh
...@@ -86,6 +86,6 @@ gitlab-runner register \ ...@@ -86,6 +86,6 @@ gitlab-runner register \
--docker-postgres latest --docker-postgres latest
``` ```
With the command above, you create a runner that uses the [`python:3.5`](https://hub.docker.com/r/_/python/) image and uses a [PostgreSQL](https://hub.docker.com/r/_/postgres/) database. With the command above, you create a runner that uses the [`python:3.5`](https://hub.docker.com/_/python) image and uses a [PostgreSQL](https://hub.docker.com/_/postgres) database.
To access the PostgreSQL database, connect to `host: postgres` as user `postgres` with no password. To access the PostgreSQL database, connect to `host: postgres` as user `postgres` with no password.
...@@ -77,6 +77,6 @@ gitlab-runner register \ ...@@ -77,6 +77,6 @@ gitlab-runner register \
--docker-postgres latest --docker-postgres latest
``` ```
With the command above, you create a Runner that uses the [ruby:2.2](https://hub.docker.com/r/_/ruby/) image and uses a [postgres](https://hub.docker.com/r/_/postgres/) database. With the command above, you create a Runner that uses the [ruby:2.2](https://hub.docker.com/_/ruby) image and uses a [postgres](https://hub.docker.com/_/postgres) database.
To access the PostgreSQL database, connect to `host: postgres` as user `postgres` with no password. To access the PostgreSQL database, connect to `host: postgres` as user `postgres` with no password.
...@@ -30,7 +30,7 @@ Host: redis ...@@ -30,7 +30,7 @@ Host: redis
And that's it. Redis will now be available to be used within your testing And that's it. Redis will now be available to be used within your testing
framework. framework.
You can also use any other docker image available on [Docker Hub][hub-redis]. You can also use any other docker image available on [Docker Hub](https://hub.docker.com/_/redis).
For example, to use Redis 2.8 the service becomes `redis:2.8`. For example, to use Redis 2.8 the service becomes `redis:2.8`.
## Use Redis with the Shell executor ## Use Redis with the Shell executor
...@@ -62,12 +62,9 @@ Host: localhost ...@@ -62,12 +62,9 @@ Host: localhost
## Example project ## Example project
We have set up an [Example Redis Project][redis-example-repo] for your convenience We have set up an [Example Redis Project](https://gitlab.com/gitlab-examples/redis) for your convenience
that runs on [GitLab.com](https://gitlab.com) using our publicly available that runs on [GitLab.com](https://gitlab.com) using our publicly available
[shared runners](../runners/README.md). [shared runners](../runners/README.md).
Want to hack on it? Simply fork it, commit and push your changes. Within a few Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the job will begin. moments the changes will be picked by a public runner and the job will begin.
[hub-redis]: https://hub.docker.com/r/_/redis/
[redis-example-repo]: https://gitlab.com/gitlab-examples/redis
...@@ -25,10 +25,10 @@ with any type of [executor](https://docs.gitlab.com/runner/executors/) ...@@ -25,10 +25,10 @@ with any type of [executor](https://docs.gitlab.com/runner/executors/)
## How it works ## How it works
1. Create a new SSH key pair locally with [`ssh-keygen`](http://linux.die.net/man/1/ssh-keygen) 1. Create a new SSH key pair locally with [`ssh-keygen`](https://linux.die.net/man/1/ssh-keygen)
1. Add the private key as a [variable](../variables/README.md) to 1. Add the private key as a [variable](../variables/README.md) to
your project your project
1. Run the [`ssh-agent`](http://linux.die.net/man/1/ssh-agent) during job to load 1. Run the [`ssh-agent`](https://linux.die.net/man/1/ssh-agent) during job to load
the private key. the private key.
1. Copy the public key to the servers you want to have access to (usually in 1. Copy the public key to the servers you want to have access to (usually in
`~/.ssh/authorized_keys`) or add it as a [deploy key](../../ssh/README.md#deploy-keys) `~/.ssh/authorized_keys`) or add it as a [deploy key](../../ssh/README.md#deploy-keys)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment