• Yorick Peterse's avatar
    Replace the changelog regex parser with Parslet · d82571bf
    Yorick Peterse authored
    In https://gitlab.com/gitlab-org/gitlab/-/merge_requests/50063 we
    introduced code for generating Markdown changelogs using the API, using
    a custom template language. The compiler for this language would convert
    a custom syntax into ERB, making sure arbitrary code execution isn't
    possible; or so we thought. In
    https://gitlab.com/gitlab-org/gitlab/-/issues/300224 we found a way to
    bypass the template engine's harness, and run arbitrary Ruby code.
    
    In response to this issue, I decided to investigate replacing the setup
    with something more secure. We always planned on doing so when deemed
    necessary, unfortunately that need arrived sooner than expected.
    
    In this commit we replace the regex/ERB based setup with a parser built
    using Parslet (http://kschiess.github.io/parslet/). Parslet makes it
    pretty easy to write a parser, and was already an indirect dependency of
    GitLab (through the license_finder Gem). This new parser doesn't allow
    for arbitrary code execution, doesn't depend on ERB, and is less fragile
    compared to the old setup. Templates are executed by walking and
    evaluating the AST nodes the parser produces. While this won't break any
    speed records, it's easy to maintain and understand, and fast enough for
    our needs.
    
    In this new setup there is a slight difference compared to the old
    setup. In the old setup, expression tags on their own line don't add a
    new line. So this:
    
        foo
        {% if something %}{% end %}
        bar
    
    Compiles into this:
    
        foo
        bar
    
    Getting this right using the Parslet parser proved difficult, so we took
    a slightly different approach: any newline following an expression tag
    (if, else, end, and each) consumes the newline that directly follows it
    (if any). This requires only a small change in the template, is easy to
    implement, and still intuitive to the user.
    
    As part of this commit we also fix a small bug that would lead to empty
    entries being included in the template, and add a note about using the
    right YAML syntax to preserve newlines.
    d82571bf
ast_spec.rb 6.49 KB