Commit cfaac753 authored by drew cimino's avatar drew cimino

&& and || operators for CI Pipeline expressions.

Refactored regex pattern matching to eagerly return tokens

Packaged behind a default-enabled feature flag and added operator documentation.
parent 54cc3b64
---
title: Add support for && and || to CI Pipeline Expressions. Change CI variable expression matching for Lexeme::Pattern to eagerly return tokens.
merge_request: 27925
author: Martin Manelli
type: added
......@@ -490,6 +490,7 @@ Below you can find supported syntax reference:
1. Equality matching using a string
> Example: `$VARIABLE == "some value"`
> Example: `$VARIABLE != "some value"` _(added in 11.11)_
You can use equality operator `==` or `!=` to compare a variable content to a
......@@ -500,6 +501,7 @@ Below you can find supported syntax reference:
1. Checking for an undefined value
> Example: `$VARIABLE == null`
> Example: `$VARIABLE != null` _(added in 11.11)_
It sometimes happens that you want to check whether a variable is defined
......@@ -510,6 +512,7 @@ Below you can find supported syntax reference:
1. Checking for an empty variable
> Example: `$VARIABLE == ""`
> Example: `$VARIABLE != ""` _(added in 11.11)_
If you want to check whether a variable is defined, but is empty, you can
......@@ -519,6 +522,7 @@ Below you can find supported syntax reference:
1. Comparing two variables
> Example: `$VARIABLE_1 == $VARIABLE_2`
> Example: `$VARIABLE_1 != $VARIABLE_2` _(added in 11.11)_
It is possible to compare two variables. This is going to compare values
......@@ -538,6 +542,7 @@ Below you can find supported syntax reference:
1. Pattern matching _(added in 11.0)_
> Example: `$VARIABLE =~ /^content.*/`
> Example: `$VARIABLE_1 !~ /^content.*/` _(added in 11.11)_
It is possible perform pattern matching against a variable and regular
......@@ -547,6 +552,19 @@ Below you can find supported syntax reference:
Pattern matching is case-sensitive by default. Use `i` flag modifier, like
`/pattern/i` to make a pattern case-insensitive.
1. Conjunction / Disjunction
> Example: `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 == "something"`
> Example: `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 =~ /thing$/ && $VARIABLE3`
> Example: `$VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/ && $VARIABLE3`
It is possible to join multiple conditions using `&&` or `||`. Any of the otherwise
supported syntax may be used in a conjunctive or disjunctive statement.
Precedence of operators follows standard Ruby 2.5 operation
[precedence](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html).
## Debug tracing
> Introduced in GitLab Runner 1.7.
......
# frozen_string_literal: true
module Gitlab
module Ci
module Pipeline
module Expression
module Lexeme
class And < Lexeme::Operator
PATTERN = /&&/.freeze
def evaluate(variables = {})
@left.evaluate(variables) && @right.evaluate(variables)
end
def self.build(_value, behind, ahead)
new(behind, ahead)
end
def self.precedence
11 # See: https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html
end
end
end
end
end
end
end
......@@ -15,10 +15,14 @@ module Gitlab
end
def self.scan(scanner)
if scanner.scan(self::PATTERN)
if scanner.scan(pattern)
Expression::Token.new(scanner.matched, self)
end
end
def self.pattern
self::PATTERN
end
end
end
end
......
......@@ -8,11 +8,6 @@ module Gitlab
class Equals < Lexeme::Operator
PATTERN = /==/.freeze
def initialize(left, right)
@left = left
@right = right
end
def evaluate(variables = {})
@left.evaluate(variables) == @right.evaluate(variables)
end
......@@ -20,6 +15,10 @@ module Gitlab
def self.build(_value, behind, ahead)
new(behind, ahead)
end
def self.precedence
10 # See: https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html
end
end
end
end
......
......@@ -8,21 +8,36 @@ module Gitlab
class Matches < Lexeme::Operator
PATTERN = /=~/.freeze
def initialize(left, right)
@left = left
@right = right
end
def evaluate(variables = {})
text = @left.evaluate(variables)
regexp = @right.evaluate(variables)
regexp.scan(text.to_s).any?
if ci_variables_complex_expressions?
# return offset of first match, or nil if no matches
if match = regexp.scan(text.to_s).first
text.to_s.index(match)
end
else
# return true or false
regexp.scan(text.to_s).any?
end
end
def self.build(_value, behind, ahead)
new(behind, ahead)
end
def self.precedence
10 # See: https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html
end
private
def ci_variables_complex_expressions?
Feature.enabled?(:ci_variables_complex_expressions, default_enabled: true)
end
end
end
end
......
......@@ -8,11 +8,6 @@ module Gitlab
class NotEquals < Lexeme::Operator
PATTERN = /!=/.freeze
def initialize(left, right)
@left = left
@right = right
end
def evaluate(variables = {})
@left.evaluate(variables) != @right.evaluate(variables)
end
......@@ -20,6 +15,10 @@ module Gitlab
def self.build(_value, behind, ahead)
new(behind, ahead)
end
def self.precedence
10 # See: https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html
end
end
end
end
......
......@@ -8,11 +8,6 @@ module Gitlab
class NotMatches < Lexeme::Operator
PATTERN = /\!~/.freeze
def initialize(left, right)
@left = left
@right = right
end
def evaluate(variables = {})
text = @left.evaluate(variables)
regexp = @right.evaluate(variables)
......@@ -23,6 +18,10 @@ module Gitlab
def self.build(_value, behind, ahead)
new(behind, ahead)
end
def self.precedence
10 # See: https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html
end
end
end
end
......
......@@ -6,9 +6,29 @@ module Gitlab
module Expression
module Lexeme
class Operator < Lexeme::Base
# This operator class is design to handle single operators that take two
# arguments. Expression::Parser was originally designed to read infix operators,
# and so the two operands are called "left" and "right" here. If we wish to
# implement an Operator that takes a greater or lesser number of arguments, a
# structural change or additional Operator superclass will likely be needed.
OperatorError = Class.new(Expression::ExpressionError)
def initialize(left, right)
raise OperatorError, 'Invalid left operand' unless left.respond_to? :evaluate
raise OperatorError, 'Invalid right operand' unless right.respond_to? :evaluate
@left = left
@right = right
end
def self.type
:operator
end
def self.precedence
raise NotImplementedError
end
end
end
end
......
# frozen_string_literal: true
module Gitlab
module Ci
module Pipeline
module Expression
module Lexeme
class Or < Lexeme::Operator
PATTERN = /\|\|/.freeze
def evaluate(variables = {})
@left.evaluate(variables) || @right.evaluate(variables)
end
def self.build(_value, behind, ahead)
new(behind, ahead)
end
def self.precedence
12 # See: https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html
end
end
end
end
end
end
end
......@@ -9,9 +9,10 @@ module Gitlab
class Pattern < Lexeme::Value
PATTERN = %r{^/.+/[ismU]*$}.freeze
NEW_PATTERN = %r{^\/([^\/]|\\/)+[^\\]\/[ismU]*}.freeze
def initialize(regexp)
@value = regexp
@value = self.class.eager_matching_with_escape_characters? ? regexp.gsub(/\\\//, '/') : regexp
unless Gitlab::UntrustedRegexp::RubySyntax.valid?(@value)
raise Lexer::SyntaxError, 'Invalid regular expression!'
......@@ -24,9 +25,17 @@ module Gitlab
raise Expression::RuntimeError, 'Invalid regular expression!'
end
def self.pattern
eager_matching_with_escape_characters? ? NEW_PATTERN : PATTERN
end
def self.build(string)
new(string)
end
def self.eager_matching_with_escape_characters?
Feature.enabled?(:ci_variables_complex_expressions, default_enabled: true)
end
end
end
end
......
......@@ -20,6 +20,19 @@ module Gitlab
Expression::Lexeme::NotMatches
].freeze
NEW_LEXEMES = [
Expression::Lexeme::Variable,
Expression::Lexeme::String,
Expression::Lexeme::Pattern,
Expression::Lexeme::Null,
Expression::Lexeme::Equals,
Expression::Lexeme::Matches,
Expression::Lexeme::NotEquals,
Expression::Lexeme::NotMatches,
Expression::Lexeme::And,
Expression::Lexeme::Or
].freeze
MAX_TOKENS = 100
def initialize(statement, max_tokens: MAX_TOKENS)
......@@ -45,7 +58,7 @@ module Gitlab
return tokens if @scanner.eos?
lexeme = LEXEMES.find do |type|
lexeme = available_lexemes.find do |type|
type.scan(@scanner).tap do |token|
tokens.push(token) if token.present?
end
......@@ -58,6 +71,10 @@ module Gitlab
raise Lexer::SyntaxError, 'Too many tokens!'
end
def available_lexemes
Feature.enabled?(:ci_variables_complex_expressions, default_enabled: true) ? NEW_LEXEMES : LEXEMES
end
end
end
end
......
......@@ -5,17 +5,30 @@ module Gitlab
module Pipeline
module Expression
class Parser
ParseError = Class.new(Expression::ExpressionError)
def initialize(tokens)
@tokens = tokens.to_enum
@nodes = []
end
##
# This produces a reverse descent parse tree.
#
# It currently does not support precedence of operators.
#
def tree
if Feature.enabled?(:ci_variables_complex_expressions, default_enabled: true)
rpn_parse_tree
else
reverse_descent_parse_tree
end
end
def self.seed(statement)
new(Expression::Lexer.new(statement).tokens)
end
private
# This produces a reverse descent parse tree.
# It does not support precedence of operators.
def reverse_descent_parse_tree
while token = @tokens.next
case token.type
when :operator
......@@ -32,8 +45,51 @@ module Gitlab
@nodes.last || Lexeme::Null.new
end
def self.seed(statement)
new(Expression::Lexer.new(statement).tokens)
def rpn_parse_tree
results = []
tokens_rpn.each do |token|
case token.type
when :value
results.push(token.build)
when :operator
right_operand = results.pop
left_operand = results.pop
token.build(left_operand, right_operand).tap do |res|
results.push(res)
end
else
raise ParseError, 'Unprocessable token found in parse tree'
end
end
raise ParseError, 'Unreachable nodes in parse tree' if results.count > 1
raise ParseError, 'Empty parse tree' if results.count < 1
results.pop
end
# Parse the expression into Reverse Polish Notation
# (See: Shunting-yard algorithm)
def tokens_rpn
output = []
operators = []
@tokens.each do |token|
case token.type
when :value
output.push(token)
when :operator
if operators.any? && token.lexeme.precedence >= operators.last.lexeme.precedence
output.push(operators.pop)
end
operators.push(token)
end
end
output.concat(operators.reverse)
end
end
end
......
......@@ -7,27 +7,6 @@ module Gitlab
class Statement
StatementError = Class.new(Expression::ExpressionError)
GRAMMAR = [
# presence matchers
%w[variable],
# positive matchers
%w[variable equals string],
%w[variable equals variable],
%w[variable equals null],
%w[string equals variable],
%w[null equals variable],
%w[variable matches pattern],
# negative matchers
%w[variable notequals string],
%w[variable notequals variable],
%w[variable notequals null],
%w[string notequals variable],
%w[null notequals variable],
%w[variable notmatches pattern]
].freeze
def initialize(statement, variables = {})
@lexer = Expression::Lexer.new(statement)
@variables = variables.with_indifferent_access
......@@ -36,10 +15,6 @@ module Gitlab
def parse_tree
raise StatementError if @lexer.lexemes.empty?
unless GRAMMAR.find { |syntax| syntax == @lexer.lexemes }
raise StatementError, 'Unknown pipeline expression!'
end
Expression::Parser.new(@lexer.tokens).tree
end
......@@ -54,6 +29,7 @@ module Gitlab
end
def valid?
evaluate
parse_tree.is_a?(Lexeme::Base)
rescue Expression::ExpressionError
false
......
require 'fast_spec_helper'
require 'rspec-parameterized'
describe Gitlab::Ci::Pipeline::Expression::Lexeme::And do
let(:left) { double('left', evaluate: nil) }
let(:right) { double('right', evaluate: nil) }
describe '.build' do
it 'creates a new instance of the token' do
expect(described_class.build('&&', left, right)).to be_a(described_class)
end
context 'with non-evaluable operands' do
let(:left) { double('left') }
let(:right) { double('right') }
it 'raises an operator error' do
expect { described_class.build('&&', left, right) }.to raise_error Gitlab::Ci::Pipeline::Expression::Lexeme::Operator::OperatorError
end
end
end
describe '.type' do
it 'is an operator' do
expect(described_class.type).to eq :operator
end
end
describe '.precedence' do
it 'has a precedence' do
expect(described_class.precedence).to be_an Integer
end
end
describe '#evaluate' do
let(:operator) { described_class.new(left, right) }
subject { operator.evaluate }
before do
allow(left).to receive(:evaluate).and_return(left_value)
allow(right).to receive(:evaluate).and_return(right_value)
end
context 'when left and right are truthy' do
where(:left_value, :right_value) do
[true, 1, 'a'].permutation(2).to_a
end
with_them do
it { is_expected.to be_truthy }
it { is_expected.to eq(right_value) }
end
end
context 'when left or right is falsey' do
where(:left_value, :right_value) do
[true, false, nil].permutation(2).to_a
end
with_them do
it { is_expected.to be_falsey }
end
end
context 'when left and right are falsey' do
where(:left_value, :right_value) do
[false, nil].permutation(2).to_a
end
with_them do
it { is_expected.to be_falsey }
it { is_expected.to eq(left_value) }
end
end
end
end
......@@ -5,11 +5,23 @@ describe Gitlab::Ci::Pipeline::Expression::Lexeme::Equals do
let(:right) { double('right') }
describe '.build' do
context 'with non-evaluable operands' do
it 'creates a new instance of the token' do
expect { described_class.build('==', left, right) }
.to raise_error Gitlab::Ci::Pipeline::Expression::Lexeme::Operator::OperatorError
end
end
context 'with evaluable operands' do
it 'creates a new instance of the token' do
allow(left).to receive(:evaluate).and_return('my-string')
allow(right).to receive(:evaluate).and_return('my-string')
expect(described_class.build('==', left, right))
.to be_a(described_class)
end
end
end
describe '.type' do
it 'is an operator' do
......@@ -17,23 +29,40 @@ describe Gitlab::Ci::Pipeline::Expression::Lexeme::Equals do
end
end
describe '.precedence' do
it 'has a precedence' do
expect(described_class.precedence).to be_an Integer
end
end
describe '#evaluate' do
it 'returns false when left and right are not equal' do
allow(left).to receive(:evaluate).and_return(1)
allow(right).to receive(:evaluate).and_return(2)
let(:operator) { described_class.new(left, right) }
operator = described_class.new(left, right)
subject { operator.evaluate }
expect(operator.evaluate(VARIABLE: 3)).to eq false
before do
allow(left).to receive(:evaluate).and_return(left_value)
allow(right).to receive(:evaluate).and_return(right_value)
end
it 'returns true when left and right are equal' do
allow(left).to receive(:evaluate).and_return(1)
allow(right).to receive(:evaluate).and_return(1)
context 'when left and right are equal' do
where(:left_value, :right_value) do
[%w(string string)]
end
operator = described_class.new(left, right)
with_them do
it { is_expected.to eq(true) }
end
end
expect(operator.evaluate(VARIABLE: 3)).to eq true
context 'when left and right are not equal' do
where(:left_value, :right_value) do
['one string', 'two string'].permutation(2).to_a
end
with_them do
it { is_expected.to eq(false) }
end
end
end
end
......@@ -6,11 +6,23 @@ describe Gitlab::Ci::Pipeline::Expression::Lexeme::Matches do
let(:right) { double('right') }
describe '.build' do
context 'with non-evaluable operands' do
it 'creates a new instance of the token' do
expect { described_class.build('=~', left, right) }
.to raise_error Gitlab::Ci::Pipeline::Expression::Lexeme::Operator::OperatorError
end
end
context 'with evaluable operands' do
it 'creates a new instance of the token' do
allow(left).to receive(:evaluate).and_return('my-string')
allow(right).to receive(:evaluate).and_return('/my-string/')
expect(described_class.build('=~', left, right))
.to be_a(described_class)
end
end
end
describe '.type' do
it 'is an operator' do
......@@ -18,63 +30,93 @@ describe Gitlab::Ci::Pipeline::Expression::Lexeme::Matches do
end
end
describe '.precedence' do
it 'has a precedence' do
expect(described_class.precedence).to be_an Integer
end
end
describe '#evaluate' do
it 'returns false when left and right do not match' do
allow(left).to receive(:evaluate).and_return('my-string')
allow(right).to receive(:evaluate)
.and_return(Gitlab::UntrustedRegexp.new('something'))
let(:operator) { described_class.new(left, right) }
operator = described_class.new(left, right)
subject { operator.evaluate }
expect(operator.evaluate).to eq false
before do
allow(left).to receive(:evaluate).and_return(left_value)
allow(right).to receive(:evaluate).and_return(right_value)
end
it 'returns true when left and right match' do
allow(left).to receive(:evaluate).and_return('my-awesome-string')
allow(right).to receive(:evaluate)
.and_return(Gitlab::UntrustedRegexp.new('awesome.string$'))
operator = described_class.new(left, right)
context 'when left and right do not match' do
let(:left_value) { 'my-string' }
let(:right_value) { Gitlab::UntrustedRegexp.new('something') }
expect(operator.evaluate).to eq true
it { is_expected.to eq(nil) }
end
it 'supports matching against a nil value' do
allow(left).to receive(:evaluate).and_return(nil)
allow(right).to receive(:evaluate)
.and_return(Gitlab::UntrustedRegexp.new('pattern'))
context 'when left and right match' do
let(:left_value) { 'my-awesome-string' }
let(:right_value) { Gitlab::UntrustedRegexp.new('awesome.string$') }
operator = described_class.new(left, right)
it { is_expected.to eq(3) }
end
context 'when left is nil' do
let(:left_value) { nil }
let(:right_value) { Gitlab::UntrustedRegexp.new('pattern') }
expect(operator.evaluate).to eq false
it { is_expected.to eq(nil) }
end
it 'supports multiline strings' do
allow(left).to receive(:evaluate).and_return <<~TEXT
context 'when left is a multiline string and matches right' do
let(:left_value) do
<<~TEXT
My awesome contents
My-text-string!
TEXT
end
allow(right).to receive(:evaluate)
.and_return(Gitlab::UntrustedRegexp.new('text-string'))
let(:right_value) { Gitlab::UntrustedRegexp.new('text-string') }
operator = described_class.new(left, right)
it { is_expected.to eq(24) }
end
context 'when left is a multiline string and does not match right' do
let(:left_value) do
<<~TEXT
My awesome contents
My-terrible-string!
TEXT
end
let(:right_value) { Gitlab::UntrustedRegexp.new('text-string') }
expect(operator.evaluate).to eq true
it { is_expected.to eq(nil) }
end
it 'supports regexp flags' do
allow(left).to receive(:evaluate).and_return <<~TEXT
context 'when a matching pattern uses regex flags' do
let(:left_value) do
<<~TEXT
My AWESOME content
TEXT
end
let(:right_value) { Gitlab::UntrustedRegexp.new('(?i)awesome') }
it { is_expected.to eq(3) }
end
allow(right).to receive(:evaluate)
.and_return(Gitlab::UntrustedRegexp.new('(?i)awesome'))
context 'when a non-matching pattern uses regex flags' do
let(:left_value) do
<<~TEXT
My AWESOME content
TEXT
end
operator = described_class.new(left, right)
let(:right_value) { Gitlab::UntrustedRegexp.new('(?i)terrible') }
expect(operator.evaluate).to eq true
it { is_expected.to eq(nil) }
end
end
end
......@@ -5,11 +5,23 @@ describe Gitlab::Ci::Pipeline::Expression::Lexeme::NotEquals do
let(:right) { double('right') }
describe '.build' do
context 'with non-evaluable operands' do
it 'creates a new instance of the token' do
expect { described_class.build('!=', left, right) }
.to raise_error Gitlab::Ci::Pipeline::Expression::Lexeme::Operator::OperatorError
end
end
context 'with evaluable operands' do
it 'creates a new instance of the token' do
allow(left).to receive(:evaluate).and_return('my-string')
allow(right).to receive(:evaluate).and_return('my-string')
expect(described_class.build('!=', left, right))
.to be_a(described_class)
end
end
end
describe '.type' do
it 'is an operator' do
......@@ -17,23 +29,45 @@ describe Gitlab::Ci::Pipeline::Expression::Lexeme::NotEquals do
end
end
describe '.precedence' do
it 'has a precedence' do
expect(described_class.precedence).to be_an Integer
end
end
describe '#evaluate' do
it 'returns true when left and right are not equal' do
allow(left).to receive(:evaluate).and_return(1)
allow(right).to receive(:evaluate).and_return(2)
let(:operator) { described_class.new(left, right) }
subject { operator.evaluate }
before do
allow(left).to receive(:evaluate).and_return(left_value)
allow(right).to receive(:evaluate).and_return(right_value)
end
operator = described_class.new(left, right)
context 'when left and right are equal' do
using RSpec::Parameterized::TableSyntax
expect(operator.evaluate(VARIABLE: 3)).to eq true
where(:left_value, :right_value) do
'string' | 'string'
1 | 1
'' | ''
nil | nil
end
it 'returns false when left and right are equal' do
allow(left).to receive(:evaluate).and_return(1)
allow(right).to receive(:evaluate).and_return(1)
with_them do
it { is_expected.to eq(false) }
end
end
operator = described_class.new(left, right)
context 'when left and right are not equal' do
where(:left_value, :right_value) do
['one string', 'two string', 1, 2, '', nil, false, true].permutation(2).to_a
end
expect(operator.evaluate(VARIABLE: 3)).to eq false
with_them do
it { is_expected.to eq(true) }
end
end
end
end
......@@ -6,11 +6,23 @@ describe Gitlab::Ci::Pipeline::Expression::Lexeme::NotMatches do
let(:right) { double('right') }
describe '.build' do
context 'with non-evaluable operands' do
it 'creates a new instance of the token' do
expect { described_class.build('!~', left, right) }
.to raise_error Gitlab::Ci::Pipeline::Expression::Lexeme::Operator::OperatorError
end
end
context 'with evaluable operands' do
it 'creates a new instance of the token' do
allow(left).to receive(:evaluate).and_return('my-string')
allow(right).to receive(:evaluate).and_return('my-string')
expect(described_class.build('!~', left, right))
.to be_a(described_class)
end
end
end
describe '.type' do
it 'is an operator' do
......@@ -18,63 +30,93 @@ describe Gitlab::Ci::Pipeline::Expression::Lexeme::NotMatches do
end
end
describe '.precedence' do
it 'has a precedence' do
expect(described_class.precedence).to be_an Integer
end
end
describe '#evaluate' do
it 'returns true when left and right do not match' do
allow(left).to receive(:evaluate).and_return('my-string')
allow(right).to receive(:evaluate)
.and_return(Gitlab::UntrustedRegexp.new('something'))
let(:operator) { described_class.new(left, right) }
operator = described_class.new(left, right)
subject { operator.evaluate }
expect(operator.evaluate).to eq true
before do
allow(left).to receive(:evaluate).and_return(left_value)
allow(right).to receive(:evaluate).and_return(right_value)
end
it 'returns false when left and right match' do
allow(left).to receive(:evaluate).and_return('my-awesome-string')
allow(right).to receive(:evaluate)
.and_return(Gitlab::UntrustedRegexp.new('awesome.string$'))
operator = described_class.new(left, right)
context 'when left and right do not match' do
let(:left_value) { 'my-string' }
let(:right_value) { Gitlab::UntrustedRegexp.new('something') }
expect(operator.evaluate).to eq false
it { is_expected.to eq(true) }
end
it 'supports matching against a nil value' do
allow(left).to receive(:evaluate).and_return(nil)
allow(right).to receive(:evaluate)
.and_return(Gitlab::UntrustedRegexp.new('pattern'))
context 'when left and right match' do
let(:left_value) { 'my-awesome-string' }
let(:right_value) { Gitlab::UntrustedRegexp.new('awesome.string$') }
operator = described_class.new(left, right)
it { is_expected.to eq(false) }
end
context 'when left is nil' do
let(:left_value) { nil }
let(:right_value) { Gitlab::UntrustedRegexp.new('pattern') }
expect(operator.evaluate).to eq true
it { is_expected.to eq(true) }
end
it 'supports multiline strings' do
allow(left).to receive(:evaluate).and_return <<~TEXT
context 'when left is a multiline string and matches right' do
let(:left_value) do
<<~TEXT
My awesome contents
My-text-string!
TEXT
end
allow(right).to receive(:evaluate)
.and_return(Gitlab::UntrustedRegexp.new('text-string'))
let(:right_value) { Gitlab::UntrustedRegexp.new('text-string') }
operator = described_class.new(left, right)
it { is_expected.to eq(false) }
end
context 'when left is a multiline string and does not match right' do
let(:left_value) do
<<~TEXT
My awesome contents
My-terrible-string!
TEXT
end
let(:right_value) { Gitlab::UntrustedRegexp.new('text-string') }
expect(operator.evaluate).to eq false
it { is_expected.to eq(true) }
end
it 'supports regexp flags' do
allow(left).to receive(:evaluate).and_return <<~TEXT
context 'when a matching pattern uses regex flags' do
let(:left_value) do
<<~TEXT
My AWESOME content
TEXT
end
let(:right_value) { Gitlab::UntrustedRegexp.new('(?i)awesome') }
it { is_expected.to eq(false) }
end
allow(right).to receive(:evaluate)
.and_return(Gitlab::UntrustedRegexp.new('(?i)awesome'))
context 'when a non-matching pattern uses regex flags' do
let(:left_value) do
<<~TEXT
My AWESOME content
TEXT
end
operator = described_class.new(left, right)
let(:right_value) { Gitlab::UntrustedRegexp.new('(?i)terrible') }
expect(operator.evaluate).to eq false
it { is_expected.to eq(true) }
end
end
end
require 'fast_spec_helper'
require 'rspec-parameterized'
describe Gitlab::Ci::Pipeline::Expression::Lexeme::Or do
let(:left) { double('left', evaluate: nil) }
let(:right) { double('right', evaluate: nil) }
describe '.build' do
it 'creates a new instance of the token' do
expect(described_class.build('||', left, right)).to be_a(described_class)
end
context 'with non-evaluable operands' do
let(:left) { double('left') }
let(:right) { double('right') }
it 'raises an operator error' do
expect { described_class.build('||', left, right) }.to raise_error Gitlab::Ci::Pipeline::Expression::Lexeme::Operator::OperatorError
end
end
end
describe '.type' do
it 'is an operator' do
expect(described_class.type).to eq :operator
end
end
describe '.precedence' do
it 'has a precedence' do
expect(described_class.precedence).to be_an Integer
end
end
describe '#evaluate' do
let(:operator) { described_class.new(left, right) }
subject { operator.evaluate }
before do
allow(left).to receive(:evaluate).and_return(left_value)
allow(right).to receive(:evaluate).and_return(right_value)
end
context 'when left and right are truthy' do
where(:left_value, :right_value) do
[true, 1, 'a'].permutation(2).to_a
end
with_them do
it { is_expected.to be_truthy }
it { is_expected.to eq(left_value) }
end
end
context 'when left or right is truthy' do
where(:left_value, :right_value) do
[true, false, 'a'].permutation(2).to_a
end
with_them do
it { is_expected.to be_truthy }
end
end
context 'when left and right are falsey' do
where(:left_value, :right_value) do
[false, nil].permutation(2).to_a
end
with_them do
it { is_expected.to be_falsey }
it { is_expected.to eq(right_value) }
end
end
end
end
require 'fast_spec_helper'
require 'spec_helper'
describe Gitlab::Ci::Pipeline::Expression::Lexeme::Pattern do
describe '.build' do
......@@ -30,16 +30,6 @@ describe Gitlab::Ci::Pipeline::Expression::Lexeme::Pattern do
.to eq Gitlab::UntrustedRegexp.new('pattern')
end
it 'is a greedy scanner for regexp boundaries' do
scanner = StringScanner.new('/some .* / pattern/')
token = described_class.scan(scanner)
expect(token).not_to be_nil
expect(token.build.evaluate)
.to eq Gitlab::UntrustedRegexp.new('some .* / pattern')
end
it 'does not allow to use an empty pattern' do
scanner = StringScanner.new(%(//))
......@@ -68,12 +58,90 @@ describe Gitlab::Ci::Pipeline::Expression::Lexeme::Pattern do
.to eq Gitlab::UntrustedRegexp.new('(?im)pattern')
end
it 'does not support arbitrary flags' do
it 'ignores unsupported flags' do
scanner = StringScanner.new('/pattern/x')
token = described_class.scan(scanner)
expect(token).to be_nil
expect(token).not_to be_nil
expect(token.build.evaluate)
.to eq Gitlab::UntrustedRegexp.new('pattern')
end
it 'is a eager scanner for regexp boundaries' do
scanner = StringScanner.new('/some .* / pattern/')
token = described_class.scan(scanner)
expect(token).not_to be_nil
expect(token.build.evaluate)
.to eq Gitlab::UntrustedRegexp.new('some .* ')
end
it 'does not match on escaped regexp boundaries' do
scanner = StringScanner.new('/some .* \/ pattern/')
token = described_class.scan(scanner)
expect(token).not_to be_nil
expect(token.build.evaluate)
.to eq Gitlab::UntrustedRegexp.new('some .* / pattern')
end
it 'recognizes \ as an escape character for /' do
scanner = StringScanner.new('/some numeric \/$ pattern/')
token = described_class.scan(scanner)
expect(token).not_to be_nil
expect(token.build.evaluate)
.to eq Gitlab::UntrustedRegexp.new('some numeric /$ pattern')
end
it 'does not recognize \ as an escape character for $' do
scanner = StringScanner.new('/some numeric \$ pattern/')
token = described_class.scan(scanner)
expect(token).not_to be_nil
expect(token.build.evaluate)
.to eq Gitlab::UntrustedRegexp.new('some numeric \$ pattern')
end
context 'with the ci_variables_complex_expressions feature flag disabled' do
before do
stub_feature_flags(ci_variables_complex_expressions: false)
end
it 'is a greedy scanner for regexp boundaries' do
scanner = StringScanner.new('/some .* / pattern/')
token = described_class.scan(scanner)
expect(token).not_to be_nil
expect(token.build.evaluate)
.to eq Gitlab::UntrustedRegexp.new('some .* / pattern')
end
it 'does not recognize the \ escape character for /' do
scanner = StringScanner.new('/some .* \/ pattern/')
token = described_class.scan(scanner)
expect(token).not_to be_nil
expect(token.build.evaluate)
.to eq Gitlab::UntrustedRegexp.new('some .* \/ pattern')
end
it 'does not recognize the \ escape character for $' do
scanner = StringScanner.new('/some numeric \$ pattern/')
token = described_class.scan(scanner)
expect(token).not_to be_nil
expect(token.build.evaluate)
.to eq Gitlab::UntrustedRegexp.new('some numeric \$ pattern')
end
end
end
......
......@@ -58,6 +58,56 @@ describe Gitlab::Ci::Pipeline::Expression::Lexer do
expect { lexer.tokens }
.to raise_error described_class::SyntaxError
end
context 'with complex expressions' do
using RSpec::Parameterized::TableSyntax
subject { described_class.new(expression).tokens.map(&:value) }
where(:expression, :tokens) do
'$PRESENT_VARIABLE =~ /my var/ && $EMPTY_VARIABLE =~ /nope/' | ['$PRESENT_VARIABLE', '=~', '/my var/', '&&', '$EMPTY_VARIABLE', '=~', '/nope/']
'$EMPTY_VARIABLE == "" && $PRESENT_VARIABLE' | ['$EMPTY_VARIABLE', '==', '""', '&&', '$PRESENT_VARIABLE']
'$EMPTY_VARIABLE == "" && $PRESENT_VARIABLE != "nope"' | ['$EMPTY_VARIABLE', '==', '""', '&&', '$PRESENT_VARIABLE', '!=', '"nope"']
'$PRESENT_VARIABLE && $EMPTY_VARIABLE' | ['$PRESENT_VARIABLE', '&&', '$EMPTY_VARIABLE']
'$PRESENT_VARIABLE =~ /my var/ || $EMPTY_VARIABLE =~ /nope/' | ['$PRESENT_VARIABLE', '=~', '/my var/', '||', '$EMPTY_VARIABLE', '=~', '/nope/']
'$EMPTY_VARIABLE == "" || $PRESENT_VARIABLE' | ['$EMPTY_VARIABLE', '==', '""', '||', '$PRESENT_VARIABLE']
'$EMPTY_VARIABLE == "" || $PRESENT_VARIABLE != "nope"' | ['$EMPTY_VARIABLE', '==', '""', '||', '$PRESENT_VARIABLE', '!=', '"nope"']
'$PRESENT_VARIABLE || $EMPTY_VARIABLE' | ['$PRESENT_VARIABLE', '||', '$EMPTY_VARIABLE']
'$PRESENT_VARIABLE && null || $EMPTY_VARIABLE == ""' | ['$PRESENT_VARIABLE', '&&', 'null', '||', '$EMPTY_VARIABLE', '==', '""']
end
with_them do
it { is_expected.to eq(tokens) }
end
end
context 'with the ci_variables_complex_expressions feature flag turned off' do
before do
stub_feature_flags(ci_variables_complex_expressions: false)
end
it 'incorrectly tokenizes conjunctive match statements as one match statement' do
tokens = described_class.new('$PRESENT_VARIABLE =~ /my var/ && $EMPTY_VARIABLE =~ /nope/').tokens
expect(tokens.map(&:value)).to eq(['$PRESENT_VARIABLE', '=~', '/my var/ && $EMPTY_VARIABLE =~ /nope/'])
end
it 'incorrectly tokenizes disjunctive match statements as one statement' do
tokens = described_class.new('$PRESENT_VARIABLE =~ /my var/ || $EMPTY_VARIABLE =~ /nope/').tokens
expect(tokens.map(&:value)).to eq(['$PRESENT_VARIABLE', '=~', '/my var/ || $EMPTY_VARIABLE =~ /nope/'])
end
it 'raises an error about && operators' do
expect { described_class.new('$EMPTY_VARIABLE == "" && $PRESENT_VARIABLE').tokens }
.to raise_error(Gitlab::Ci::Pipeline::Expression::Lexer::SyntaxError).with_message('Unknown lexeme found!')
end
it 'raises an error about || operators' do
expect { described_class.new('$EMPTY_VARIABLE == "" || $PRESENT_VARIABLE').tokens }
.to raise_error(Gitlab::Ci::Pipeline::Expression::Lexer::SyntaxError).with_message('Unknown lexeme found!')
end
end
end
describe '#lexemes' do
......
......@@ -2,25 +2,67 @@ require 'fast_spec_helper'
describe Gitlab::Ci::Pipeline::Expression::Parser do
describe '#tree' do
context 'when using operators' do
context 'when using two operators' do
it 'returns a reverse descent parse tree' do
expect(described_class.seed('$VAR1 == "123"').tree)
.to be_a Gitlab::Ci::Pipeline::Expression::Lexeme::Equals
end
end
context 'when using three operators' do
it 'returns a reverse descent parse tree' do
expect(described_class.seed('$VAR1 == "123" == $VAR2').tree)
.to be_a Gitlab::Ci::Pipeline::Expression::Lexeme::Equals
end
end
context 'when using a single token' do
context 'when using a single variable token' do
it 'returns a single token instance' do
expect(described_class.seed('$VAR').tree)
.to be_a Gitlab::Ci::Pipeline::Expression::Lexeme::Variable
end
end
context 'when using a single string token' do
it 'returns a single token instance' do
expect(described_class.seed('"some value"').tree)
.to be_a Gitlab::Ci::Pipeline::Expression::Lexeme::String
end
end
context 'when expression is empty' do
it 'returns a null token' do
expect(described_class.seed('').tree)
expect { described_class.seed('').tree }
.to raise_error Gitlab::Ci::Pipeline::Expression::Parser::ParseError
end
end
context 'when expression is null' do
it 'returns a null token' do
expect(described_class.seed('null').tree)
.to be_a Gitlab::Ci::Pipeline::Expression::Lexeme::Null
end
end
context 'when two value tokens have no operator' do
it 'raises a parsing error' do
expect { described_class.seed('$VAR "text"').tree }
.to raise_error Gitlab::Ci::Pipeline::Expression::Parser::ParseError
end
end
context 'when an operator has no left side' do
it 'raises an OperatorError' do
expect { described_class.seed('== "123"').tree }
.to raise_error Gitlab::Ci::Pipeline::Expression::Lexeme::Operator::OperatorError
end
end
context 'when an operator has no right side' do
it 'raises an OperatorError' do
expect { described_class.seed('$VAR ==').tree }
.to raise_error Gitlab::Ci::Pipeline::Expression::Lexeme::Operator::OperatorError
end
end
end
end
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment