mysqlslap has an unusual semantics of the --iterations option, so that
$ mysqlslap --concurrency=16 --iterations=1000
performs 16,000 connections. Because of that it gets many times slower
with --ssl

Let's disable ssl for mysqlslap in mysql-test