Skip to content

G
gitlab-ce
Commit 50297719 authored by Fabio Pitino's avatar Fabio Pitino Committed by Shinya Maeda
Browse files
Options

Port Ci::Bridge to Core 

Move most of the model and specs
parent f503bfec
Hide whitespace changes
Inline Side-by-side
Showing with 680 additions and 748 deletions
changelogs/unreleased/port-trigger-keyword-to-ce.yml 0 → 100644
View file @ 50297719
---
title: Port `trigger` keyword in CI config to Core
merge_request: 24191
author:
type: fixed
doc/ci/multi_project_pipelines.md
View file @ 50297719
......@@ -51,7 +51,8 @@ outbound connections for upstream and downstream pipeline dependencies.
## Creating multi-project pipelines from `.gitlab-ci.yml`
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/issues/199224) to GitLab Core in 12.8.
### Triggering a downstream pipeline using a bridge job
......@@ -181,7 +182,8 @@ the ones defined in the upstream project will take precedence.
### Mirroring status from triggered pipeline
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/11238) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.3.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/11238) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.3.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/issues/199224) to GitLab Core in 12.8.
You can mirror the pipeline status from the triggered pipeline to the source
bridge job by using `strategy: depend`. For example:
......
doc/ci/parent_child_pipelines.md
View file @ 50297719
......@@ -45,7 +45,7 @@ the child pipeline configuration.
## Examples
The simplest case is [triggering a child pipeline](yaml/README.md#trigger-premium) using a
The simplest case is [triggering a child pipeline](yaml/README.md#trigger) using a
local YAML file to define the pipeline configuration. In this case, the parent pipeline will
trigger the child pipeline, and continue without waiting:
......
doc/ci/yaml/README.md
View file @ 50297719
......@@ -113,7 +113,7 @@ The following table lists available parameters for jobs:
| [`retry`](#retry) | When and how many times a job can be auto-retried in case of a failure. |
| [`timeout`](#timeout) | Define a custom job-level timeout that takes precedence over the project-wide setting. |
| [`parallel`](#parallel) | How many instances of a job should be run in parallel. |
| [`trigger`](#trigger-premium) | Defines a downstream pipeline trigger. |
| [`trigger`](#trigger) | Defines a downstream pipeline trigger. |
| [`include`](#include) | Allows this job to include external YAML files. Also available: `include:local`, `include:file`, `include:template`, and `include:remote`. |
| [`extends`](#extends) | Configuration entries that this job is going to inherit from. |
| [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. |
......@@ -2572,9 +2572,10 @@ Please be aware that semaphore_test_boosters reports usages statistics to the au
You can then navigate to the **Jobs** tab of a new pipeline build and see your RSpec
job split into three separate jobs.
### `trigger` **(PREMIUM)**
### `trigger`
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/issues/199224) to GitLab Core in 12.8.
`trigger` allows you to define downstream pipeline trigger. When a job created
from `trigger` definition is started by GitLab, a downstream pipeline gets
......@@ -3892,7 +3893,7 @@ job_no_git_strategy:
Triggers can be used to force a rebuild of a specific branch, tag or commit,
with an API call when a pipeline gets created using a trigger token.
Not to be confused with [`trigger`](#trigger-premium).
Not to be confused with [`trigger`](#trigger).
[Read more in the triggers documentation.](../triggers/README.md)
......
ee/lib/ee/gitlab/ci/config/entry/bridge.rb deleted 100644 → 0
View file @ f503bfec
# frozen_string_literal: true
module EE
module Gitlab
module Ci
module Config
module Entry
##
# Entry that represents a CI/CD Bridge job that is responsible for
# defining a downstream project trigger.
#
class Bridge < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Configurable
include ::Gitlab::Config::Entry::Attributable
include ::Gitlab::Config::Entry::Inheritable
ALLOWED_KEYS = %i[trigger stage allow_failure only except
when extends variables needs rules].freeze
validations do
validates :config, allowed_keys: ALLOWED_KEYS
validates :config, presence: true
validates :name, presence: true
validates :name, type: Symbol
validates :config, disallowed_keys: {
in: %i[only except when start_in],
message: 'key may not be used with `rules`'
},
if: :has_rules?
with_options allow_nil: true do
validates :when,
inclusion: { in: %w[on_success on_failure always],
message: 'should be on_success, on_failure or always' }
validates :extends, type: String
validates :rules, array_of_hashes: true
end
validate on: :composed do
unless trigger.present? || bridge_needs.present?
errors.add(:config, 'should contain either a trigger or a needs:pipeline')
end
end
validate on: :composed do
next unless bridge_needs.present?
next if bridge_needs.one?
errors.add(:config, 'should contain at most one bridge need')
end
end
entry :trigger, ::EE::Gitlab::Ci::Config::Entry::Trigger,
description: 'CI/CD Bridge downstream trigger definition.',
inherit: false
entry :needs, ::Gitlab::Ci::Config::Entry::Needs,
description: 'CI/CD Bridge needs dependency definition.',
inherit: false,
metadata: { allowed_needs: %i[bridge job] }
entry :stage, ::Gitlab::Ci::Config::Entry::Stage,
description: 'Pipeline stage this job will be executed into.',
inherit: false
entry :only, ::Gitlab::Ci::Config::Entry::Policy,
description: 'Refs policy this job will be executed for.',
default: ::Gitlab::Ci::Config::Entry::Policy::DEFAULT_ONLY,
inherit: false
entry :except, ::Gitlab::Ci::Config::Entry::Policy,
description: 'Refs policy this job will be executed for.',
inherit: false
entry :rules, ::Gitlab::Ci::Config::Entry::Rules,
description: 'List of evaluable Rules to determine job inclusion.',
inherit: false,
metadata: {
allowed_when: %w[on_success on_failure always never manual delayed].freeze
}
entry :variables, ::Gitlab::Ci::Config::Entry::Variables,
description: 'Environment variables available for this job.',
inherit: false
helpers(*ALLOWED_KEYS)
attributes(*ALLOWED_KEYS)
def self.matching?(name, config)
::Feature.enabled?(:cross_project_pipeline_triggers, default_enabled: true) &&
!name.to_s.start_with?('.') &&
config.is_a?(Hash) &&
(config.key?(:trigger) || config.key?(:needs))
end
def self.visible?
true
end
def compose!(deps = nil)
super do
has_workflow_rules = deps&.workflow&.has_rules?
# If workflow:rules: or rules: are used
# they are considered not compatible
# with `only/except` defaults
#
# Context: https://gitlab.com/gitlab-org/gitlab/merge_requests/21742
if has_rules? || has_workflow_rules
# Remove only/except defaults
# defaults are not considered as defined
@entries.delete(:only) unless only_defined?
@entries.delete(:except) unless except_defined?
end
end
end
def has_rules?
@config&.key?(:rules)
end
def name
@metadata[:name]
end
def value
{ name: name,
trigger: (trigger_value if trigger_defined?),
needs: (needs_value if needs_defined?),
ignore: !!allow_failure,
stage: stage_value,
when: when_value,
extends: extends_value,
variables: (variables_value if variables_defined?),
rules: (rules_value if has_rules?),
only: only_value,
except: except_value }.compact
end
def bridge_needs
needs_value[:bridge] if needs_value
end
private
def overwrite_entry(deps, key, current_entry)
deps.default[key] unless current_entry.specified?
end
end
end
end
end
end
end
ee/lib/ee/gitlab/ci/config/entry/jobs.rb deleted 100644 → 0
View file @ f503bfec
# frozen_string_literal: true
module EE
module Gitlab
module Ci
module Config
module Entry
module Jobs
extend ActiveSupport::Concern
prepended do
EE_TYPES = const_get(:TYPES, false) + [::EE::Gitlab::Ci::Config::Entry::Bridge]
end
class_methods do
extend ::Gitlab::Utils::Override
override :all_types
def all_types
EE_TYPES
end
end
end
end
end
end
end
end
ee/lib/ee/gitlab/ci/config/entry/need.rb
View file @ 50297719
......@@ -9,10 +9,13 @@ module EE
extend ActiveSupport::Concern
prepended do
# When defining a bridge that subscribes to an upstream pipeline:
# needs:pipeline: other/project
strategy :BridgeHash,
class: EE::Gitlab::Ci::Config::Entry::Need::BridgeHash,
if: -> (config) { config.is_a?(Hash) && !config.key?(:job) && !config.key?(:project) }
# When defining DAG dependency across project/ref
strategy :CrossDependency,
class: EE::Gitlab::Ci::Config::Entry::Need::CrossDependency,
if: -> (config) { config.is_a?(Hash) && (config.key?(:project) || config.key?(:ref)) }
......
ee/lib/ee/gitlab/ci/config/entry/trigger.rb deleted 100644 → 0
View file @ f503bfec
# frozen_string_literal: true
module EE
module Gitlab
module Ci
module Config
module Entry
##
# Entry that represents a cross-project downstream trigger.
#
class Trigger < ::Gitlab::Config::Entry::Simplifiable
strategy :SimpleTrigger, if: -> (config) { config.is_a?(String) }
strategy :ComplexTrigger, if: -> (config) { config.is_a?(Hash) }
class SimpleTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
validations { validates :config, presence: true }
def value
{ project: @config }
end
end
class ComplexTrigger < ::Gitlab::Config::Entry::Simplifiable
strategy :CrossProjectTrigger, if: -> (config) { !config.key?(:include) }
strategy :SameProjectTrigger, if: -> (config) do
::Feature.enabled?(:ci_parent_child_pipeline, default_enabled: true) &&
config.key?(:include)
end
class CrossProjectTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
include ::Gitlab::Config::Entry::Attributable
ALLOWED_KEYS = %i[project branch strategy].freeze
attributes :project, :branch, :strategy
validations do
validates :config, presence: true
validates :config, allowed_keys: ALLOWED_KEYS
validates :project, presence: true
validates :branch, type: String, allow_nil: true
validates :strategy, type: String, inclusion: { in: %w[depend], message: 'should be depend' }, allow_nil: true
end
end
class SameProjectTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
include ::Gitlab::Config::Entry::Attributable
include ::Gitlab::Config::Entry::Configurable
INCLUDE_MAX_SIZE = 3
ALLOWED_KEYS = %i[strategy include].freeze
attributes :strategy
validations do
validates :config, presence: true
validates :config, allowed_keys: ALLOWED_KEYS
validates :strategy, type: String, inclusion: { in: %w[depend], message: 'should be depend' }, allow_nil: true
end
entry :include, ::Gitlab::Ci::Config::Entry::Includes,
description: 'List of external YAML files to include.',
reserved: true,
metadata: { max_size: INCLUDE_MAX_SIZE }
def value
@config
end
end
class UnknownStrategy < ::Gitlab::Config::Entry::Node
def errors
if ::Feature.enabled?(:ci_parent_child_pipeline, default_enabled: true)
['config must specify either project or include']
else
['config must specify project']
end
end
end
end
class UnknownStrategy < ::Gitlab::Config::Entry::Node
def errors
["#{location} has to be either a string or a hash"]
end
end
end
end
end
end
end
end
ee/spec/lib/ee/gitlab/ci/config/entry/bridge_spec.rb
View file @ 50297719
......@@ -2,52 +2,12 @@
require 'spec_helper'
describe EE::Gitlab::Ci::Config::Entry::Bridge do
describe Gitlab::Ci::Config::Entry::Bridge do
subject { described_class.new(config, name: :my_bridge) }
it_behaves_like 'with inheritable CI config' do
let(:inheritable_key) { 'default' }
let(:inheritable_class) { Gitlab::Ci::Config::Entry::Default }
# These are entries defined in Default
# that we know that we don't want to inherit
# as they do not have sense in context of Bridge
let(:ignored_inheritable_columns) do
%i[before_script after_script image services cache interruptible timeout
retry tags artifacts]
end
end
describe '.matching?' do
subject { described_class.matching?(name, config) }
context 'when config is not a hash' do
let(:name) { :my_trigger }
let(:config) { 'string' }
it { is_expected.to be_falsey }
end
context 'when config is a regular job' do
let(:name) { :my_trigger }
let(:config) do
{ script: 'ls -al' }
end
it { is_expected.to be_falsey }
context 'with rules' do
let(:config) do
{
script: 'ls -al',
rules: [{ if: '$VAR == "value"', when: 'always' }]
}
end
it { is_expected.to be_falsey }
end
end
context 'when config is a bridge job' do
let(:name) { :my_trigger }
let(:config) do
......@@ -55,26 +15,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
end
it { is_expected.to be_truthy }
context 'with rules' do
let(:config) do
{
trigger: 'other-project',
rules: [{ if: '$VAR == "value"', when: 'always' }]
}
end
it { is_expected.to be_truthy }
end
end
context 'when config is a hidden job' do
let(:name) { '.my_trigger' }
let(:config) do
{ trigger: 'other-project' }
end
it { is_expected.to be_falsey }
end
end
......@@ -93,24 +33,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
}
end
context 'when trigger config is a non-empty string' do
let(:config) { { trigger: 'some/project' } }
describe '#valid?' do
it { is_expected.to be_valid }
end
describe '#value' do
it 'is returns a bridge job configuration' do
expect(subject.value).to eq(name: :my_bridge,
trigger: { project: 'some/project' },
ignore: false,
stage: 'test',
only: { refs: %w[branches tags] })
end
end
end
context 'when needs pipeline config is a non-empty string' do
let(:config) { { needs: { pipeline: 'some/project' } } }
......@@ -129,27 +51,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
end
end
context 'when bridge trigger is a hash' do
let(:config) do
{ trigger: { project: 'some/project', branch: 'feature' } }
end
describe '#valid?' do
it { is_expected.to be_valid }
end
describe '#value' do
it 'is returns a bridge job configuration hash' do
expect(subject.value).to eq(name: :my_bridge,
trigger: { project: 'some/project',
branch: 'feature' },
ignore: false,
stage: 'test',
only: { refs: %w[branches tags] })
end
end
end
context 'when bridge configuration contains trigger, needs, when, extends, stage, only, except, and variables' do
let(:config) do
base_config.merge({
......@@ -162,45 +63,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
it { is_expected.to be_valid }
end
context 'when bridge configuration uses rules' do
let(:config) { base_config.merge({ rules: [{ if: '$VAR == null', when: 'never' }] }) }
it { is_expected.to be_valid }
end
context 'when bridge configuration uses rules with job:when' do
let(:config) do
base_config.merge({
when: 'always',
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when bridge configuration uses rules with only' do
let(:config) do
base_config.merge({
only: { variables: %w[$SOMEVARIABLE] },
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when bridge configuration uses rules with except' do
let(:config) do
base_config.merge({
except: { refs: %w[feature] },
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when trigger config is nil' do
let(:config) { { trigger: nil } }
......@@ -229,43 +91,6 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
end
end
context 'when bridge has only job needs' do
let(:config) do
{
needs: ['some_job']
}
end
describe '#valid?' do
it { is_expected.not_to be_valid }
end
end
context 'when bridge has only cross projects dependencies' do
let(:config) do
{
needs: [
{
project: 'some/project',
job: 'some/job',
ref: 'some/ref',
artifacts: true
}
]
}
end
describe '#valid?' do
it { is_expected.not_to be_valid }
end
describe '#errors' do
it 'returns an error about cross dependencies' do
expect(subject.errors).to include('needs config uses invalid types: cross_dependency')
end
end
end
context 'when bridge has bridge and job needs' do
let(:config) do
{
......@@ -324,35 +149,5 @@ describe EE::Gitlab::Ci::Config::Entry::Bridge do
end
end
end
context 'when bridge config contains unknown keys' do
let(:config) { { unknown: 123 } }
describe '#valid?' do
it { is_expected.not_to be_valid }
end
describe '#errors' do
it 'is returns an error about unknown config key' do
expect(subject.errors.first)
.to match /config contains unknown keys: unknown/
end
end
end
context 'when bridge config contains build-specific attributes' do
let(:config) { { script: 'something' } }
describe '#valid?' do
it { is_expected.not_to be_valid }
end
describe '#errors' do
it 'returns an error message' do
expect(subject.errors.first)
.to match /contains unknown keys: script/
end
end
end
end
end
ee/spec/lib/gitlab/ci/config/entry/jobs_spec.rb deleted 100644 → 0
View file @ f503bfec
# frozen_string_literal: true
require 'spec_helper'
describe Gitlab::Ci::Config::Entry::Jobs do
let(:config) do
{
'.hidden_job'.to_sym => { script: 'something' },
'.hidden_bridge'.to_sym => { trigger: 'my/project' },
regular_job: { script: 'something' },
my_trigger: { trigger: 'my/project' }
}
end
describe '.all_types' do
subject { described_class.all_types }
it { is_expected.to include(::EE::Gitlab::Ci::Config::Entry::Bridge) }
end
describe '.find_type' do
using RSpec::Parameterized::TableSyntax
subject { described_class.find_type(name, config[name]) }
context 'when cross-project pipeline triggers are enabled' do
before do
stub_feature_flags(cross_project_pipeline_triggers: true)
end
where(:name, :type) do
:'.hidden_job' | ::Gitlab::Ci::Config::Entry::Hidden
:'.hidden_bridge' | ::Gitlab::Ci::Config::Entry::Hidden
:regular_job | ::Gitlab::Ci::Config::Entry::Job
:my_trigger | ::EE::Gitlab::Ci::Config::Entry::Bridge
end
with_them do
it { is_expected.to eq(type) }
end
end
context 'when cross-project pipeline triggers are disabled' do
before do
stub_feature_flags(cross_project_pipeline_triggers: false)
end
where(:name, :type) do
:'.hidden_job' | ::Gitlab::Ci::Config::Entry::Hidden
:'.hidden_bridge' | ::Gitlab::Ci::Config::Entry::Hidden
:regular_job | ::Gitlab::Ci::Config::Entry::Job
:my_trigger | nil
end
with_them do
it { is_expected.to eq(type) }
end
end
end
describe '.new' do
subject do
described_class.new(config)
end
context 'when cross-project pipeline triggers are enabled' do
before do
stub_feature_flags(cross_project_pipeline_triggers: true)
subject.compose!
end
describe '#valid?' do
it { is_expected.to be_valid }
end
describe '#value' do
it 'returns a correct hash representing all jobs' do
expect(subject.value).to eq(
my_trigger: {
name: :my_trigger,
trigger: { project: 'my/project' },
stage: 'test',
only: { refs: %w[branches tags] },
ignore: false
},
regular_job: {
script: %w[something],
name: :regular_job,
stage: 'test',
only: { refs: %w[branches tags] },
variables: {},
ignore: false
})
end
end
end
context 'when cross-project pipeline triggers are disabled' do
before do
stub_feature_flags(cross_project_pipeline_triggers: false)
subject.compose!
end
describe '#valid?' do
it { is_expected.not_to be_valid }
end
end
end
end
ee/spec/services/ci/create_pipeline_service_spec.rb
View file @ 50297719
......@@ -142,112 +142,6 @@ describe Ci::CreatePipelineService, '#execute' do
end
end
describe 'child pipeline triggers' do
before do
stub_ci_pipeline_yaml_file <<~YAML
test:
script: rspec
deploy:
variables:
CROSS: downstream
stage: deploy
trigger:
include:
- local: path/to/child.yml
YAML
end
it 'creates bridge jobs correctly' do
pipeline = create_pipeline!
test = pipeline.statuses.find_by(name: 'test')
bridge = pipeline.statuses.find_by(name: 'deploy')
expect(pipeline).to be_persisted
expect(test).to be_a Ci::Build
expect(bridge).to be_a Ci::Bridge
expect(bridge.stage).to eq 'deploy'
expect(pipeline.statuses).to match_array [test, bridge]
expect(bridge.options).to eq(
'trigger' => { 'include' => [{ 'local' => 'path/to/child.yml' }] }
)
expect(bridge.yaml_variables)
.to include(key: 'CROSS', value: 'downstream', public: true)
end
end
describe 'child pipeline triggers' do
context 'when YAML is valid' do
before do
stub_ci_pipeline_yaml_file <<~YAML
test:
script: rspec
deploy:
variables:
CROSS: downstream
stage: deploy
trigger:
include:
- local: path/to/child.yml
YAML
end
it 'creates bridge jobs correctly' do
pipeline = create_pipeline!
test = pipeline.statuses.find_by(name: 'test')
bridge = pipeline.statuses.find_by(name: 'deploy')
expect(pipeline).to be_persisted
expect(test).to be_a Ci::Build
expect(bridge).to be_a Ci::Bridge
expect(bridge.stage).to eq 'deploy'
expect(pipeline.statuses).to match_array [test, bridge]
expect(bridge.options).to eq(
'trigger' => { 'include' => [{ 'local' => 'path/to/child.yml' }] }
)
expect(bridge.yaml_variables)
.to include(key: 'CROSS', value: 'downstream', public: true)
end
end
context 'when YAML is invalid' do
let(:config) do
{
test: { script: 'rspec' },
deploy: {
trigger: { include: included_files }
}
}
end
let(:included_files) do
Array.new(include_max_size + 1) do |index|
{ local: "file#{index}.yml" }
end
end
let(:include_max_size) do
EE::Gitlab::Ci::Config::Entry::Trigger::ComplexTrigger::SameProjectTrigger::INCLUDE_MAX_SIZE
end
before do
stub_ci_pipeline_yaml_file(YAML.dump(config))
end
it 'returns errors' do
pipeline = create_pipeline!
expect(pipeline.errors.full_messages.first).to match(/trigger:include config is too long/)
expect(pipeline.failure_reason).to eq 'config_error'
expect(pipeline).to be_persisted
expect(pipeline.status).to eq 'failed'
end
end
end
def create_pipeline!
service.execute(:push)
end
......
lib/gitlab/ci/config/entry/bridge.rb 0 → 100644
View file @ 50297719
# frozen_string_literal: true
module Gitlab
module Ci
class Config
module Entry
##
# Entry that represents a CI/CD Bridge job that is responsible for
# defining a downstream project trigger.
#
class Bridge < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Configurable
include ::Gitlab::Config::Entry::Attributable
include ::Gitlab::Config::Entry::Inheritable
ALLOWED_KEYS = %i[trigger stage allow_failure only except
when extends variables needs rules].freeze
validations do
validates :config, allowed_keys: ALLOWED_KEYS
validates :config, presence: true
validates :name, presence: true
validates :name, type: Symbol
validates :config, disallowed_keys: {
in: %i[only except when start_in],
message: 'key may not be used with `rules`'
},
if: :has_rules?
with_options allow_nil: true do
validates :when,
inclusion: { in: %w[on_success on_failure always],
message: 'should be on_success, on_failure or always' }
validates :extends, type: String
validates :rules, array_of_hashes: true
end
validate on: :composed do
unless trigger.present? || bridge_needs.present?
errors.add(:config, 'should contain either a trigger or a needs:pipeline')
end
end
validate on: :composed do
next unless bridge_needs.present?
next if bridge_needs.one?
errors.add(:config, 'should contain at most one bridge need')
end
end
entry :trigger, ::Gitlab::Ci::Config::Entry::Trigger,
description: 'CI/CD Bridge downstream trigger definition.',
inherit: false
entry :needs, ::Gitlab::Ci::Config::Entry::Needs,
description: 'CI/CD Bridge needs dependency definition.',
inherit: false,
metadata: { allowed_needs: %i[job bridge] }
entry :stage, ::Gitlab::Ci::Config::Entry::Stage,
description: 'Pipeline stage this job will be executed into.',
inherit: false
entry :only, ::Gitlab::Ci::Config::Entry::Policy,
description: 'Refs policy this job will be executed for.',
default: ::Gitlab::Ci::Config::Entry::Policy::DEFAULT_ONLY,
inherit: false
entry :except, ::Gitlab::Ci::Config::Entry::Policy,
description: 'Refs policy this job will be executed for.',
inherit: false
entry :rules, ::Gitlab::Ci::Config::Entry::Rules,
description: 'List of evaluable Rules to determine job inclusion.',
inherit: false,
metadata: {
allowed_when: %w[on_success on_failure always never manual delayed].freeze
}
entry :variables, ::Gitlab::Ci::Config::Entry::Variables,
description: 'Environment variables available for this job.',
inherit: false
helpers(*ALLOWED_KEYS)
attributes(*ALLOWED_KEYS)
def self.matching?(name, config)
!name.to_s.start_with?('.') &&
config.is_a?(Hash) &&
(config.key?(:trigger) || config.key?(:needs))
end
def self.visible?
true
end
def compose!(deps = nil)
super do
has_workflow_rules = deps&.workflow&.has_rules?
# If workflow:rules: or rules: are used
# they are considered not compatible
# with `only/except` defaults
#
# Context: https://gitlab.com/gitlab-org/gitlab/merge_requests/21742
if has_rules? || has_workflow_rules
# Remove only/except defaults
# defaults are not considered as defined
@entries.delete(:only) unless only_defined?
@entries.delete(:except) unless except_defined?
end
end
end
def has_rules?
@config&.key?(:rules)
end
def name
@metadata[:name]
end
def value
{ name: name,
trigger: (trigger_value if trigger_defined?),
needs: (needs_value if needs_defined?),
ignore: !!allow_failure,
stage: stage_value,
when: when_value,
extends: extends_value,
variables: (variables_value if variables_defined?),
rules: (rules_value if has_rules?),
only: only_value,
except: except_value }.compact
end
def bridge_needs
needs_value[:bridge] if needs_value
end
private
def overwrite_entry(deps, key, current_entry)
deps.default[key] unless current_entry.specified?
end
end
end
end
end
end
lib/gitlab/ci/config/entry/jobs.rb
View file @ 50297719
......@@ -36,7 +36,7 @@ module Gitlab
end
end
TYPES = [Entry::Hidden, Entry::Job].freeze
TYPES = [Entry::Hidden, Entry::Job, Entry::Bridge].freeze
private_constant :TYPES
......@@ -77,5 +77,3 @@ module Gitlab
end
end
end
::Gitlab::Ci::Config::Entry::Jobs.prepend_if_ee('::EE::Gitlab::Ci::Config::Entry::Jobs')
lib/gitlab/ci/config/entry/trigger.rb 0 → 100644
View file @ 50297719
# frozen_string_literal: true
module Gitlab
module Ci
class Config
module Entry
##
# Entry that represents a cross-project downstream trigger.
#
class Trigger < ::Gitlab::Config::Entry::Simplifiable
strategy :SimpleTrigger, if: -> (config) { config.is_a?(String) }
strategy :ComplexTrigger, if: -> (config) { config.is_a?(Hash) }
class SimpleTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
validations { validates :config, presence: true }
def value
{ project: @config }
end
end
class ComplexTrigger < ::Gitlab::Config::Entry::Simplifiable
strategy :CrossProjectTrigger, if: -> (config) { !config.key?(:include) }
strategy :SameProjectTrigger, if: -> (config) do
::Feature.enabled?(:ci_parent_child_pipeline, default_enabled: true) &&
config.key?(:include)
end
class CrossProjectTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
include ::Gitlab::Config::Entry::Attributable
ALLOWED_KEYS = %i[project branch strategy].freeze
attributes :project, :branch, :strategy
validations do
validates :config, presence: true
validates :config, allowed_keys: ALLOWED_KEYS
validates :project, presence: true
validates :branch, type: String, allow_nil: true
validates :strategy, type: String, inclusion: { in: %w[depend], message: 'should be depend' }, allow_nil: true
end
end
class SameProjectTrigger < ::Gitlab::Config::Entry::Node
include ::Gitlab::Config::Entry::Validatable
include ::Gitlab::Config::Entry::Attributable
include ::Gitlab::Config::Entry::Configurable
INCLUDE_MAX_SIZE = 3
ALLOWED_KEYS = %i[strategy include].freeze
attributes :strategy
validations do
validates :config, presence: true
validates :config, allowed_keys: ALLOWED_KEYS
validates :strategy, type: String, inclusion: { in: %w[depend], message: 'should be depend' }, allow_nil: true
end
entry :include, ::Gitlab::Ci::Config::Entry::Includes,
description: 'List of external YAML files to include.',
reserved: true,
metadata: { max_size: INCLUDE_MAX_SIZE }
def value
@config
end
end
class UnknownStrategy < ::Gitlab::Config::Entry::Node
def errors
if ::Feature.enabled?(:ci_parent_child_pipeline, default_enabled: true)
['config must specify either project or include']
else
['config must specify project']
end
end
end
end
class UnknownStrategy < ::Gitlab::Config::Entry::Node
def errors
["#{location} has to be either a string or a hash"]
end
end
end
end
end
end
end
spec/lib/gitlab/ci/config/entry/bridge_spec.rb 0 → 100644
View file @ 50297719
# frozen_string_literal: true
require 'spec_helper'
describe Gitlab::Ci::Config::Entry::Bridge do
subject { described_class.new(config, name: :my_bridge) }
it_behaves_like 'with inheritable CI config' do
let(:inheritable_key) { 'default' }
let(:inheritable_class) { Gitlab::Ci::Config::Entry::Default }
# These are entries defined in Default
# that we know that we don't want to inherit
# as they do not have sense in context of Bridge
let(:ignored_inheritable_columns) do
%i[before_script after_script image services cache interruptible timeout
retry tags artifacts]
end
end
describe '.matching?' do
subject { described_class.matching?(name, config) }
context 'when config is not a hash' do
let(:name) { :my_trigger }
let(:config) { 'string' }
it { is_expected.to be_falsey }
end
context 'when config is a regular job' do
let(:name) { :my_trigger }
let(:config) do
{ script: 'ls -al' }
end
it { is_expected.to be_falsey }
context 'with rules' do
let(:config) do
{
script: 'ls -al',
rules: [{ if: '$VAR == "value"', when: 'always' }]
}
end
it { is_expected.to be_falsey }
end
end
context 'when config is a bridge job' do
let(:name) { :my_trigger }
let(:config) do
{ trigger: 'other-project' }
end
it { is_expected.to be_truthy }
context 'with rules' do
let(:config) do
{
trigger: 'other-project',
rules: [{ if: '$VAR == "value"', when: 'always' }]
}
end
it { is_expected.to be_truthy }
end
end
context 'when config is a hidden job' do
let(:name) { '.my_trigger' }
let(:config) do
{ trigger: 'other-project' }
end
it { is_expected.to be_falsey }
end
end
describe '.new' do
before do
subject.compose!
end
let(:base_config) do
{
trigger: { project: 'some/project', branch: 'feature' },
extends: '.some-key',
stage: 'deploy',
variables: { VARIABLE: '123' }
}
end
context 'when trigger config is a non-empty string' do
let(:config) { { trigger: 'some/project' } }
describe '#valid?' do
it { is_expected.to be_valid }
end
describe '#value' do
it 'is returns a bridge job configuration' do
expect(subject.value).to eq(name: :my_bridge,
trigger: { project: 'some/project' },
ignore: false,
stage: 'test',
only: { refs: %w[branches tags] })
end
end
end
context 'when bridge trigger is a hash' do
let(:config) do
{ trigger: { project: 'some/project', branch: 'feature' } }
end
describe '#valid?' do
it { is_expected.to be_valid }
end
describe '#value' do
it 'is returns a bridge job configuration hash' do
expect(subject.value).to eq(name: :my_bridge,
trigger: { project: 'some/project',
branch: 'feature' },
ignore: false,
stage: 'test',
only: { refs: %w[branches tags] })
end
end
end
context 'when bridge configuration contains trigger, when, extends, stage, only, except, and variables' do
let(:config) do
base_config.merge({
when: 'always',
only: { variables: %w[$SOMEVARIABLE] },
except: { refs: %w[feature] }
})
end
it { is_expected.to be_valid }
end
context 'when bridge configuration uses rules' do
let(:config) { base_config.merge({ rules: [{ if: '$VAR == null', when: 'never' }] }) }
it { is_expected.to be_valid }
end
context 'when bridge configuration uses rules with job:when' do
let(:config) do
base_config.merge({
when: 'always',
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when bridge configuration uses rules with only' do
let(:config) do
base_config.merge({
only: { variables: %w[$SOMEVARIABLE] },
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when bridge configuration uses rules with except' do
let(:config) do
base_config.merge({
except: { refs: %w[feature] },
rules: [{ if: '$VAR == null', when: 'never' }]
})
end
it { is_expected.not_to be_valid }
end
context 'when bridge has only job needs' do
let(:config) do
{
needs: ['some_job']
}
end
describe '#valid?' do
it { is_expected.not_to be_valid }
end
end
context 'when bridge config contains unknown keys' do
let(:config) { { unknown: 123 } }
describe '#valid?' do
it { is_expected.not_to be_valid }
end
describe '#errors' do
it 'is returns an error about unknown config key' do
expect(subject.errors.first)
.to match /config contains unknown keys: unknown/
end
end
end
context 'when bridge config contains build-specific attributes' do
let(:config) { { script: 'something' } }
describe '#valid?' do
it { is_expected.not_to be_valid }
end
describe '#errors' do
it 'returns an error message' do
expect(subject.errors.first)
.to match /contains unknown keys: script/
end
end
end
end
end
spec/lib/gitlab/ci/config/entry/jobs_spec.rb
View file @ 50297719
......@@ -5,27 +5,31 @@ require 'spec_helper'
describe Gitlab::Ci::Config::Entry::Jobs do
let(:entry) { described_class.new(config) }
let(:config) do
{
'.hidden_job'.to_sym => { script: 'something' },
'.hidden_bridge'.to_sym => { trigger: 'my/project' },
regular_job: { script: 'something' },
my_trigger: { trigger: 'my/project' }
}
end
describe '.all_types' do
subject { described_class.all_types }
it { is_expected.to include(::Gitlab::Ci::Config::Entry::Hidden) }
it { is_expected.to include(::Gitlab::Ci::Config::Entry::Job) }
it { is_expected.to include(::Gitlab::Ci::Config::Entry::Bridge) }
end
describe '.find_type' do
using RSpec::Parameterized::TableSyntax
let(:config) do
{
'.hidden_job'.to_sym => { script: 'something' },
regular_job: { script: 'something' },
invalid_job: 'text'
}
end
where(:name, :type) do
:'.hidden_job' | ::Gitlab::Ci::Config::Entry::Hidden
:'.hidden_bridge' | ::Gitlab::Ci::Config::Entry::Hidden
:regular_job | ::Gitlab::Ci::Config::Entry::Job
:my_trigger | ::Gitlab::Ci::Config::Entry::Bridge
:invalid_job | nil
end
......@@ -42,8 +46,6 @@ describe Gitlab::Ci::Config::Entry::Jobs do
end
context 'when entry config value is correct' do
let(:config) { { rspec: { script: 'rspec' } } }
describe '#valid?' do
it 'is valid' do
expect(entry).to be_valid
......@@ -88,43 +90,41 @@ describe Gitlab::Ci::Config::Entry::Jobs do
entry.compose!
end
let(:config) do
{ rspec: { script: 'rspec' },
spinach: { script: 'spinach' },
'.hidden'.to_sym => {} }
end
describe '#value' do
it 'returns key value' do
expect(entry.value).to eq(
rspec: { name: :rspec,
script: %w[rspec],
ignore: false,
stage: 'test',
only: { refs: %w[branches tags] },
variables: {} },
spinach: { name: :spinach,
script: %w[spinach],
ignore: false,
stage: 'test',
only: { refs: %w[branches tags] },
variables: {} })
my_trigger: {
ignore: false,
name: :my_trigger,
only: { refs: %w[branches tags] },
stage: 'test',
trigger: { project: 'my/project' }
},
regular_job: {
ignore: false,
name: :regular_job,
only: { refs: %w[branches tags] },
script: ['something'],
stage: 'test',
variables: {}
})
end
end
describe '#descendants' do
it 'creates valid descendant nodes' do
expect(entry.descendants.count).to eq 3
expect(entry.descendants.first(2))
.to all(be_an_instance_of(Gitlab::Ci::Config::Entry::Job))
expect(entry.descendants.last)
.to be_an_instance_of(Gitlab::Ci::Config::Entry::Hidden)
expect(entry.descendants.map(&:class)).to eq [
Gitlab::Ci::Config::Entry::Hidden,
Gitlab::Ci::Config::Entry::Hidden,
Gitlab::Ci::Config::Entry::Job,
Gitlab::Ci::Config::Entry::Bridge
]
end
end
describe '#value' do
it 'returns value of visible jobs only' do
expect(entry.value.keys).to eq [:rspec, :spinach]
expect(entry.value.keys).to eq [:regular_job, :my_trigger]
end
end
end
......
ee/spec/lib/ee/gitlab/ci/config/entry/trigger_spec.rb → spec/lib/gitlab/ci/config/entry/trigger_spec.rb
View file @ 50297719
......@@ -2,7 +2,7 @@
require 'spec_helper'
describe EE::Gitlab::Ci::Config::Entry::Trigger do
describe Gitlab::Ci::Config::Entry::Trigger do
subject { described_class.new(config) }
context 'when trigger config is a non-empty string' do
......
spec/models/ci/bridge_spec.rb
View file @ 50297719
......@@ -63,6 +63,26 @@ describe Ci::Bridge do
end
end
describe 'state machine transitions' do
context 'when bridge points towards downstream' do
it 'schedules downstream pipeline creation' do
expect(bridge).to receive(:schedule_downstream_pipeline!)
bridge.enqueue!
end
end
end
describe 'state machine transitions' do
context 'when bridge points towards downstream' do
it 'schedules downstream pipeline creation' do
expect(bridge).to receive(:schedule_downstream_pipeline!)
bridge.enqueue!
end
end
end
describe '#inherit_status_from_downstream!' do
let(:downstream_pipeline) { build(:ci_pipeline, status: downstream_status) }
......
spec/services/ci/create_cross_project_pipeline_service_spec.rb
View file @ 50297719
......@@ -342,9 +342,7 @@ describe Ci::CreateCrossProjectPipelineService, '#execute' do
let(:service) { described_class.new(upstream_project, upstream_project.owner) }
context 'that include the bridge job' do
# TODO: this is skipped because `trigger` keyword does not exist yet.
# enabling it in the next MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/24393
xit 'creates the downstream pipeline' do
it 'creates the downstream pipeline' do
expect { service.execute(bridge) }
.to change(downstream_project.ci_pipelines, :count).by(1)
end
......
spec/services/ci/create_pipeline_service/parent_child_pipeline_spec.rb 0 → 100644
View file @ 50297719
# frozen_string_literal: true
require 'spec_helper'
describe Ci::CreatePipelineService, '#execute' do
set(:project) { create(:project, :repository) }
set(:user) { create(:user) }
let(:ref_name) { 'master' }
let(:service) do
params = { ref: ref_name,
before: '00000000',
after: project.commit.id,
commits: [{ message: 'some commit' }] }
described_class.new(project, user, params)
end
before do
project.add_developer(user)
stub_ci_pipeline_to_return_yaml_file
end
describe 'child pipeline triggers' do
before do
stub_ci_pipeline_yaml_file <<~YAML
test:
script: rspec
deploy:
variables:
CROSS: downstream
stage: deploy
trigger:
include:
- local: path/to/child.yml
YAML
end
it 'creates bridge jobs correctly' do
pipeline = create_pipeline!
test = pipeline.statuses.find_by(name: 'test')
bridge = pipeline.statuses.find_by(name: 'deploy')
expect(pipeline).to be_persisted
expect(test).to be_a Ci::Build
expect(bridge).to be_a Ci::Bridge
expect(bridge.stage).to eq 'deploy'
expect(pipeline.statuses).to match_array [test, bridge]
expect(bridge.options).to eq(
'trigger' => { 'include' => [{ 'local' => 'path/to/child.yml' }] }
)
expect(bridge.yaml_variables)
.to include(key: 'CROSS', value: 'downstream', public: true)
end
end
describe 'child pipeline triggers' do
context 'when YAML is valid' do
before do
stub_ci_pipeline_yaml_file <<~YAML
test:
script: rspec
deploy:
variables:
CROSS: downstream
stage: deploy
trigger:
include:
- local: path/to/child.yml
YAML
end
it 'creates bridge jobs correctly' do
pipeline = create_pipeline!
test = pipeline.statuses.find_by(name: 'test')
bridge = pipeline.statuses.find_by(name: 'deploy')
expect(pipeline).to be_persisted
expect(test).to be_a Ci::Build
expect(bridge).to be_a Ci::Bridge
expect(bridge.stage).to eq 'deploy'
expect(pipeline.statuses).to match_array [test, bridge]
expect(bridge.options).to eq(
'trigger' => { 'include' => [{ 'local' => 'path/to/child.yml' }] }
)
expect(bridge.yaml_variables)
.to include(key: 'CROSS', value: 'downstream', public: true)
end
end
context 'when YAML is invalid' do
let(:config) do
{
test: { script: 'rspec' },
deploy: {
trigger: { include: included_files }
}
}
end
let(:included_files) do
Array.new(include_max_size + 1) do |index|
{ local: "file#{index}.yml" }
end
end
let(:include_max_size) do
Gitlab::Ci::Config::Entry::Trigger::ComplexTrigger::SameProjectTrigger::INCLUDE_MAX_SIZE
end
before do
stub_ci_pipeline_yaml_file(YAML.dump(config))
end
it 'returns errors' do
pipeline = create_pipeline!
expect(pipeline.errors.full_messages.first).to match(/trigger:include config is too long/)
expect(pipeline.failure_reason).to eq 'config_error'
expect(pipeline).to be_persisted
expect(pipeline.status).to eq 'failed'
end
end
end
def create_pipeline!
service.execute(:push)
end
end
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7