Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
slapos
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Eteri
slapos
Commits
71b5141f
Commit
71b5141f
authored
Apr 27, 2018
by
Eteri
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
tests fluentd-keepalive-caddy
parent
2d000220
Changes
12
Show whitespace changes
Inline
Side-by-side
Showing
12 changed files
with
397 additions
and
222 deletions
+397
-222
software/erp5testnode/testsuite/fluentTest/instance-caddy-fluentd.cfg.in
...stnode/testsuite/fluentTest/instance-caddy-fluentd.cfg.in
+33
-17
software/erp5testnode/testsuite/fluentTest/runTestSuite.in
software/erp5testnode/testsuite/fluentTest/runTestSuite.in
+0
-108
software/erp5testnode/testsuite/fluentTest/software.cfg
software/erp5testnode/testsuite/fluentTest/software.cfg
+67
-31
software/erp5testnode/testsuite/fluentTest/templates/Caddyfile.in
.../erp5testnode/testsuite/fluentTest/templates/Caddyfile.in
+4
-0
software/erp5testnode/testsuite/fluentTest/templates/empty.in
...ware/erp5testnode/testsuite/fluentTest/templates/empty.in
+0
-1
software/erp5testnode/testsuite/fluentTest/templates/fluentd-conf.cnf.in
...stnode/testsuite/fluentTest/templates/fluentd-conf.cnf.in
+5
-4
software/erp5testnode/testsuite/fluentTest/templates/index.html
...re/erp5testnode/testsuite/fluentTest/templates/index.html
+0
-1
software/erp5testnode/testsuite/fluentTest/templates/template-caddy-service.sh.in
...stsuite/fluentTest/templates/template-caddy-service.sh.in
+0
-0
software/erp5testnode/testsuite/fluentTest/tests/README.md
software/erp5testnode/testsuite/fluentTest/tests/README.md
+1
-0
software/erp5testnode/testsuite/fluentTest/tests/setup.py
software/erp5testnode/testsuite/fluentTest/tests/setup.py
+51
-0
software/erp5testnode/testsuite/fluentTest/tests/test.py
software/erp5testnode/testsuite/fluentTest/tests/test.py
+66
-60
software/erp5testnode/testsuite/fluentTest/tests/utils.py
software/erp5testnode/testsuite/fluentTest/tests/utils.py
+170
-0
No files found.
software/erp5testnode/testsuite/fluentTest/instance-caddy-fluentd.cfg.in
View file @
71b5141f
[buildout]
[buildout]
parts =
parts =
slapos-test-runner
directory
directory
runTestSuite-instance
fluentdConfig
fluentdConfig
fluentd-service
fluentd-service
caddy-service
caddy-service
...
@@ -26,43 +26,34 @@ promise = $${:etc}/promise/
...
@@ -26,43 +26,34 @@ promise = $${:etc}/promise/
www = $${:srv}/www
www = $${:srv}/www
home = $${:etc}/home
home = $${:etc}/home
ssl = $${:etc}/ssl
ssl = $${:etc}/ssl
working-dir = $${buildout:directory}/tmp/
[runTestSuite-instance]
recipe = slapos.recipe.template
url = ${template-runTestSuite:output}
output = $${directory:bin}/runTestSuite
buildout-directory = $${buildout:directory}
mode = 0700
[runTest-instance]
[runTest-instance]
recipe = slapos.recipe.template
recipe = slapos.recipe.template
url = ${template-test:output}
url = ${template-test:output}
output = $${directory:bin}/test
Ingestion
.py
output = $${directory:bin}/test.py
buildout-directory = $${buildout:directory}
buildout-directory = $${buildout:directory}
mode = 0700
mode = 0700
#################################
#################################
# fluentd service
# fluentd service
#################################
#################################
[fluentdConfig]
[fluentdConfig]
recipe = slapos.recipe.template:jinja2
recipe = slapos.recipe.template:jinja2
template = ${template-fluentdConfig:
location}/${template-fluentdConfig:filename
}
template = ${template-fluentdConfig:
output
}
rendered = $${directory:etc}/fluentd-conf.cnf
rendered = $${directory:etc}/fluentd-conf.cnf
mode = 0600
mode = 0600
context =
context =
key local_ip caddy-configuration:local_ip
key local_ip caddy-configuration:local_ip
[fluentd-service]
[fluentd-service]
recipe = slapos.cookbook:wrapper
recipe = slapos.cookbook:wrapper
wrapper-path = $${directory:services}/fluentd-service
wrapper-path = $${directory:services}/fluentd-service
path = ${fluentd:location}
path = ${fluentd:location}
command-line = ${fluentd:location}/bin/fluentd
command-line = ${fluentd:location}/bin/fluentd
-vv
-c $${directory:etc}/fluentd-conf.cnf -p ${fluentd-plugin-dev-repository:location}/lib/fluent/plugin/
-c $${directory:etc}/fluentd-conf.cnf -p ${fluentd-plugin-dev-repository:location}/lib/fluent/plugin/
-o $${directory:etc}/fluent.log
environment =
environment =
GEM_PATH= ${fluentd:location}/lib/ruby/gems/1.8/
GEM_PATH= ${fluentd:location}/lib/ruby/gems/1.8/
output = $${:wrapper-path}
output = $${:wrapper-path}
...
@@ -95,10 +86,35 @@ local_ip = $${slap-network-information:local-ipv4}
...
@@ -95,10 +86,35 @@ local_ip = $${slap-network-information:local-ipv4}
context =
context =
key local_ip caddy-configuration:local_ip
key local_ip caddy-configuration:local_ip
[publish-connection-information]
[publish-connection-information]
recipe = slapos.cookbook:publish
recipe = slapos.cookbook:publish
secure_access = http://$${caddy-configuration:local_ip}:4443}
secure_access = http://$${caddy-configuration:local_ip}:4443}
[download-source]
recipe = slapos.recipe.build:gitclone
git-executable = ${git:location}/bin/git
[slapos]
<= download-source
repository = ${slapos-repository:location}
[create-directory]
recipe = slapos.cookbook:mkdirectory
bin = $${buildout:directory}/bin
[slapos-test-runner]
recipe = slapos.cookbook:wrapper
wrapper-path = $${directory:bin}/runTestSuite
command-line =
${buildout:bin-directory}/runTestSuite
--python_interpreter=${buildout:bin-directory}/${eggs:interpreter}
--source_code_path_list=$${slapos:location}/software/erp5testnode/testsuite/fluentTest/tests/
environment =
PATH=${buildout:bin-directory}:/usr/bin/:/bin/
LOCAL_IPV4=$${slap-network-information:local-ipv4}
GLOBAL_IPV6=$${slap-network-information:global-ipv6}
FLUENT_SERVICE = $${fluentd-service:path}
SLAPOS_TEST_WORKING_DIR=$${directory:working-dir}
CADDY_DIR = $${directory:etc}/caddy_pidfile
DEBUG=1
\ No newline at end of file
software/erp5testnode/testsuite/fluentTest/runTestSuite.in
deleted
100644 → 0
View file @
2d000220
#!${buildout:directory}/bin/${eggs:interpreter}
# BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically
import argparse, os, re, shutil, subprocess, sys, traceback
from erp5.util import taskdistribution
import sys
test_path = '$${runTest-instance:buildout-directory}/bin/'
sys.path.append(test_path)
import testIngestion
def main():
parser = argparse.ArgumentParser(description='Run a test suite.')
parser.add_argument('--test_suite', help='The test suite name')
parser.add_argument('--test_suite_title', help='The test suite title')
parser.add_argument('--test_node_title', help='The test node title')
parser.add_argument('--project_title', help='The project title')
parser.add_argument('--revision', help='The revision to test',
default='dummy_revision')
parser.add_argument('--node_quantity', help='ignored', type=int)
parser.add_argument('--master_url',
help='The Url of Master controling many suites')
parser.add_argument('--frontend_url',
help='The url of frontend of the test suite')
parser.add_argument('--target',
help='Target OS to run tests on',
type=str)
parser.add_argument('--target_version',
help='Target OS version to use',
type=str,)
parser.add_argument('--target_browser',
help='The desired browser of the target OS to be used. Example: Firefox if target is Android.',
type=str,)
parser.add_argument('--target_device',
help='The desired device running the target OS. Example: iPad Simulator, if target is iOS.',
type=str,)
parser.add_argument('--appium_server_auth',
help='Combination of user and token to access SauceLabs service. (i.e. user:token)',
type=str)
args = parser.parse_args()
try:
test_suite_title = args.test_suite_title or args.test_suite
test_suite = args.test_suite
revision = args.revision
test_line_dict = {}
######!!!!!!!!!!!!!! uncomment this !!!!!!!!
# result.errors, result.failures, stream.read()
num_of_tests, errors, failures, stdout = testIngestion.main()
result = testIngestion.main()
test_line_dict["testIngestion"] = {
'test_count': num_of_tests,
'error_count': len(errors),
'failure_count': len(failures),
'skip_count': 0,
'stdout': stdout,
'stderr': errors
}
tool = taskdistribution.TaskDistributor(portal_url=args.master_url)
test_result = tool.createTestResult(revision = revision,
test_name_list = test_line_dict.keys(),
node_title = args.test_node_title,
test_title = test_suite_title,
project_title = args.project_title)
if test_result is None or not hasattr(args, 'master_url'):
print("test_result is NONE")
return
# report test results
while 1:
test_result_line = test_result.start()
if not test_result_line:
print 'No test result anymore.'
break
print 'Submitting: "%s"' % test_result_line.name
# report status back to Nexedi ERP5
test_result_line.stop(**test_line_dict[test_result_line.name])
except:
# Catch any exception here, to warn user instead of being silent,
# by generating fake error result
print traceback.format_exc()
result = dict(status_code=-1,
command=url,
stderr=traceback.format_exc(),
stdout='')
# XXX: inform test node master of error
raise EnvironmentError(result)
if __name__ == "__main__":
main()
software/erp5testnode/testsuite/fluentTest/software.cfg
View file @
71b5141f
...
@@ -14,10 +14,62 @@ parts =
...
@@ -14,10 +14,62 @@ parts =
fluentd
fluentd
fluentd-plugin-dev-repository
fluentd-plugin-dev-repository
instance-profile
instance-profile
template-runTestSuite
template-fluentdConfig
template-fluentdConfig
[setup-develop-egg]
recipe = zc.recipe.egg:develop
[slapos.test.fluentTest-setup]
<= setup-develop-egg
egg = slapos.test.fluentTest
setup = ${slapos-repository:location}/software/erp5testnode/testsuite/fluentTest/tests/
[erp5.util-setup]
<= setup-develop-egg
egg = erp5.util[testnode]
setup = ${erp5.util-repository:location}
[eggs]
recipe = zc.recipe.egg
eggs =
${slapos.test.fluentTest-setup:egg}
${erp5.util-setup:egg}
slapos.core
entry-points =
runTestSuite=erp5.util.testsuite:runTestSuite
scripts =
runTestSuite
slapos
interpreter=
python_for_test
[git-clone-repository]
recipe = slapos.recipe.build:gitclone
git-executable = ${git:location}/bin/git
forbid-download-cache = true
branch = master
[slapos-repository]
<= git-clone-repository
repository = https://lab.nexedi.com/Sokhoyan/slapos.git
branch = FluentTest_withFixUtil
# XXX we need an unreleased ( 0.4.51 ) version of erp5.util runTestSuite
# later we can stop fetching it from git and just use egg
[erp5.util-repository]
<= git-clone-repository
repository = https://lab.nexedi.com/nexedi/erp5.git
revision = 3b9d23bff23cfba208131b004ae4a0e7484e9aff
[versions]
# clear the version of tested eggs, to make sure we installed the developped ones
slapos.test.fluentTest =
erp5.util =
#erp5.util = 0.4.51
[instance-profile]
[instance-profile]
recipe = slapos.recipe.template
recipe = slapos.recipe.template
md5sum = 45c4adaae557bed5c25c24f4c4506b3d
md5sum = 45c4adaae557bed5c25c24f4c4506b3d
...
@@ -25,34 +77,18 @@ url = ${:_profile_base_location_}/instance.cfg.in
...
@@ -25,34 +77,18 @@ url = ${:_profile_base_location_}/instance.cfg.in
output = ${buildout:directory}/instance.cfg
output = ${buildout:directory}/instance.cfg
mode = 0644
mode = 0644
[template-runTestSuite]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/runTestSuite.in
#md5sum = ff66d13f73982e8257eb5535cdb541c7
output = ${buildout:directory}/runTestSuite.in
mode = 0644
[template-test]
[template-test]
recipe = slapos.recipe.template
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/test
Ingestion
.py
url = ${:_profile_base_location_}/test
s/test
.py
#md5sum = ff66d13f73982e8257eb5535cdb541c7
#md5sum = ff66d13f73982e8257eb5535cdb541c7
output = ${buildout:directory}/test
Ingestion
.py
output = ${buildout:directory}/test.py
mode = 0644
mode = 0644
[eggs]
recipe = zc.recipe.egg
eggs =
erp5.util
${lxml-python:egg}
requests
interpreter = pythonwitheggs
[template-fluentdConfig]
[template-fluentdConfig]
recipe = slapos.recipe.build:download
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/fluentd-conf.cnf.in
url = ${:_profile_base_location_}/templates/fluentd-conf.cnf.in
#md5sum = 3132b99d65c75b72fc6dacb702ddca0b
md5sum = f37d4d69895680abf40b847633876373
filename = fluentd-conf.cnf.in
output = ${buildout:directory}/fluentd-conf.cnf.in
location = ${buildout:parts-directory}/${:_buildout_section_name_}
mode = 0644
mode = 0644
[fluentd-plugin-dev-repository]
[fluentd-plugin-dev-repository]
...
@@ -71,22 +107,22 @@ gems =
...
@@ -71,22 +107,22 @@ gems =
[template-caddy-service]
[template-caddy-service]
recipe = slapos.recipe.template
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/template-caddy-service.sh.in
url = ${:_profile_base_location_}/template
s/template
-caddy-service.sh.in
md5sum = 3e09969c479931ca4df6546abbd1a117
md5sum = 3e09969c479931ca4df6546abbd1a117
output = ${buildout:directory}/template-caddy-service.sh.in
output = ${buildout:directory}/template-caddy-service.sh.in
mode = 0644
mode = 0644
[template-caddyfile]
[template-caddyfile]
recipe = slapos.recipe.template
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/Caddyfile.in
url = ${:_profile_base_location_}/
templates/
Caddyfile.in
md5sum =
3132b99d65c75b72fc6dacb702ddca0b
md5sum =
8644e98038ab76a8e5a499e22f92660a
output = ${buildout:directory}/Caddyfile.in
output = ${buildout:directory}/Caddyfile.in
mode = 0644
mode = 0644
[template-caddy-fluentd]
[template-caddy-fluentd]
recipe = slapos.recipe.template
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-caddy-fluentd.cfg.in
url = ${:_profile_base_location_}/instance-caddy-fluentd.cfg.in
md5sum =
599d4650e9b96ce15fceb512316651ae
md5sum =
22f725151611725fdca77e080b8d18e4
output = ${buildout:directory}/instance-caddy-fluentd.cfg.in
output = ${buildout:directory}/instance-caddy-fluentd.cfg.in
mode = 0644
mode = 0644
...
@@ -102,6 +138,6 @@ environment =
...
@@ -102,6 +138,6 @@ environment =
GOPATH=${gowork:directory}
GOPATH=${gowork:directory}
output = ${gowork:bin}/caddy
output = ${gowork:bin}/caddy
[
versions]
#
versions]
slapos.recipe.template = 4.3
#
slapos.recipe.template = 4.3
erp5.util = 0.4.50
#
erp5.util = 0.4.50
\ No newline at end of file
software/erp5testnode/testsuite/fluentTest/Caddyfile.in
→
software/erp5testnode/testsuite/fluentTest/
templates/
Caddyfile.in
View file @
71b5141f
{{local_ip}}:4443
{{local_ip}}:4443
proxy /
0.0.0.0
:9443
proxy /
{{local_ip}}
:9443
software/erp5testnode/testsuite/fluentTest/templates/empty.in
deleted
100644 → 0
View file @
2d000220
{{ content }}
\ No newline at end of file
software/erp5testnode/testsuite/fluentTest/fluentd-conf.cnf.in
→
software/erp5testnode/testsuite/fluentTest/
templates/
fluentd-conf.cnf.in
View file @
71b5141f
<source>
<source>
@type forward
@type forward
port 5438
</source>
</source>
#<source>
#<source>
...
@@ -19,7 +20,7 @@
...
@@ -19,7 +20,7 @@
## output tag=sensor.** to Wendelin
## output tag=sensor.** to Wendelin
<match
*
*>
<match
tag_
*>
@type wendelin
@type wendelin
@id wendelin_out
@id wendelin_out
...
@@ -57,7 +58,7 @@
...
@@ -57,7 +58,7 @@
@type monitor_agent
@type monitor_agent
@id monitor_agent_input
@id monitor_agent_input
port
2422
0
port
544
0
</source>
</source>
# Listen DRb for debug
# Listen DRb for debug
...
@@ -65,8 +66,8 @@
...
@@ -65,8 +66,8 @@
@type debug_agent
@type debug_agent
@id debug_agent_input
@id debug_agent_input
bind
127.0.0.1
bind
{{local_ip}}
port
24230
port
5443
</source>
</source>
...
...
software/erp5testnode/testsuite/fluentTest/templates/index.html
deleted
100644 → 0
View file @
2d000220
<h1>
Welcome
</h1>
\ No newline at end of file
software/erp5testnode/testsuite/fluentTest/template-caddy-service.sh.in
→
software/erp5testnode/testsuite/fluentTest/template
s/template
-caddy-service.sh.in
View file @
71b5141f
File moved
software/erp5testnode/testsuite/fluentTest/tests/README.md
0 → 100644
View file @
71b5141f
Test for fluentd wendelin plugin with keep-alive connection and caddy.
\ No newline at end of file
software/erp5testnode/testsuite/fluentTest/tests/setup.py
0 → 100644
View file @
71b5141f
##############################################################################
#
# Copyright (c) 2018 Nexedi SA and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
from
setuptools
import
setup
,
find_packages
import
glob
import
os
version
=
'0.0.1.dev0'
name
=
'slapos.test.fluentTest'
long_description
=
open
(
"README.md"
).
read
()
setup
(
name
=
name
,
version
=
version
,
description
=
"Test for fluentd wendelin plugin"
,
long_description
=
long_description
,
long_description_content_type
=
'text/markdown'
,
maintainer
=
"Nexedi"
,
maintainer_email
=
"info@nexedi.com"
,
url
=
"https://lab.nexedi.com/nexedi/slapos"
,
packages
=
find_packages
(),
install_requires
=
[
'slapos.core'
,
'erp5.util'
,
'requests'
,
],
zip_safe
=
True
,
test_suite
=
'test'
,
)
\ No newline at end of file
software/erp5testnode/testsuite/fluentTest/test
Ingestion
.py
→
software/erp5testnode/testsuite/fluentTest/test
s/test
.py
View file @
71b5141f
...
@@ -11,20 +11,30 @@ import SocketServer
...
@@ -11,20 +11,30 @@ import SocketServer
from
BaseHTTPServer
import
BaseHTTPRequestHandler
,
HTTPServer
from
BaseHTTPServer
import
BaseHTTPRequestHandler
,
HTTPServer
import
os
import
os
import
threading
import
time
import
time
import
utils
import
threading
test_msg
=
"dummyInputSimpleIngest"
test_msg
=
"dummyInputSimpleIngest"
url
=
"http://$${caddy-configuration:local_ip}:4443"
#url = "http://$${caddy-configuration:local_ip}:8443"
caddy_pidfile
=
"$${directory:etc}/caddy_pidfile"
url
=
"http://"
+
os
.
environ
.
get
(
'LOCAL_IPV4'
)
+
":4443"
#caddy_pidfile = "$${directory:etc}/caddy_pidfile"
caddy_pidfile
=
os
.
environ
.
get
(
'CADDY_DIR'
)
posted_data
=
None
posted_data
=
None
all_data
=
[]
all_data
=
[]
request_tag
=
""
request_tag
=
""
with
open
(
caddy_pidfile
)
as
f
:
caddy_pid
=
f
.
readline
()
if
os
.
environ
.
get
(
'DEBUG'
):
import
logging
logging
.
basicConfig
(
level
=
logging
.
DEBUG
)
import
unittest
unittest
.
installHandler
()
class
FluentdPluginTestCase
(
utils
.
SlapOSInstanceTestCase
):
@
classmethod
def
getSoftwareURLList
(
cls
):
return
(
os
.
path
.
abspath
(
os
.
path
.
join
(
os
.
path
.
dirname
(
__file__
),
'..'
,
'software.cfg'
)),
)
class
TestServerHandler
(
BaseHTTPRequestHandler
):
class
TestServerHandler
(
BaseHTTPRequestHandler
):
...
@@ -58,17 +68,47 @@ class TestServerHandler(BaseHTTPRequestHandler):
...
@@ -58,17 +68,47 @@ class TestServerHandler(BaseHTTPRequestHandler):
global
request_tag
global
request_tag
request_tag
=
find_tag
(
self
.
requestline
,
"="
,
" "
)
request_tag
=
find_tag
(
self
.
requestline
,
"="
,
" "
)
class
TestPost
(
unittest
.
TestCase
):
class
TestIngestion
(
FluentdPluginTestCase
):
@
classmethod
def
startServer
(
cls
):
port
=
9443
server_address
=
(
os
.
environ
.
get
(
'LOCAL_IPV4'
),
port
)
cls
.
server
=
HTTPServer
(
server_address
,
TestServerHandler
)
cls
.
thread
=
threading
.
Thread
(
target
=
cls
.
server
.
serve_forever
)
cls
.
thread
.
start
()
time
.
sleep
(
10
)
print
(
"server start"
)
posted_data
=
""
@
classmethod
def
stopServer
(
cls
):
cls
.
server
.
shutdown
()
cls
.
server
.
server_close
()
print
(
"serever shutdown"
)
time
.
sleep
(
10
)
# def setUp(self):
# self.startServer()
# def tearDown(self):
# #self.stopServer()
# time.sleep(10)
def
test_1_get
(
self
):
def
test_1_get
(
self
):
self
.
startServer
()
time
.
sleep
(
10
)
print
(
"start server"
)
print
(
"############## TEST 1 ##############"
)
print
(
"############## TEST 1 ##############"
)
resp
=
requests
.
get
(
url
)
resp
=
requests
.
get
(
url
)
self
.
assertEqual
(
resp
.
status_code
,
200
)
self
.
assertEqual
(
resp
.
status_code
,
200
)
print
(
resp
.
status_code
)
print
(
resp
.
status_code
)
def
test_2_ingest
(
self
):
def
test_2_ingest
(
self
):
time
.
sleep
(
10
)
print
(
"############## TEST 2 ##############"
)
print
(
"############## TEST 2 ##############"
)
start_fluentd_cat
(
test_msg
,
"tag_test_2"
)
start_fluentd_cat
(
test_msg
,
"tag_test_2"
)
time
.
sleep
(
10
)
time
.
sleep
(
10
)
...
@@ -79,7 +119,7 @@ class TestPost(unittest.TestCase):
...
@@ -79,7 +119,7 @@ class TestPost(unittest.TestCase):
else
:
else
:
self
.
assertEqual
(
test_msg
,
posted_data
)
self
.
assertEqual
(
test_msg
,
posted_data
)
time
.
sleep
(
10
)
time
.
sleep
(
10
)
self
.
assertEqual
(
test_msg
,
posted_data
.
split
(
" "
)[
1
])
def
test_3_keepAlive_on
(
self
):
def
test_3_keepAlive_on
(
self
):
print
(
"############## TEST 3 ##############"
)
print
(
"############## TEST 3 ##############"
)
...
@@ -90,6 +130,8 @@ class TestPost(unittest.TestCase):
...
@@ -90,6 +130,8 @@ class TestPost(unittest.TestCase):
def
test_4_delay_15_mins
(
self
):
def
test_4_delay_15_mins
(
self
):
print
(
"############## TEST 4 ##############"
)
print
(
"############## TEST 4 ##############"
)
# sleep 15mins to test that connections doesn't break after long delay
# and data is ingested correctly after the delay.
time
.
sleep
(
900
)
time
.
sleep
(
900
)
start_fluentd_cat
(
"dummyInputDelay"
,
"tag_test_4"
)
start_fluentd_cat
(
"dummyInputDelay"
,
"tag_test_4"
)
time
.
sleep
(
15
)
time
.
sleep
(
15
)
...
@@ -137,63 +179,27 @@ class TestPost(unittest.TestCase):
...
@@ -137,63 +179,27 @@ class TestPost(unittest.TestCase):
start_fluentd_cat
(
"dummyInputTags_6_3"
,
"test_Tag_6_3"
)
start_fluentd_cat
(
"dummyInputTags_6_3"
,
"test_Tag_6_3"
)
time
.
sleep
(
2
)
time
.
sleep
(
2
)
self
.
assertEqual
(
"test_Tag_6_3"
,
request_tag
)
self
.
assertEqual
(
"test_Tag_6_3"
,
request_tag
)
self
.
stopServer
()
time
.
sleep
(
10
)
def
start_fluentd_cat
(
test_msg
,
tag
):
def
start_fluentd_cat
(
test_msg
,
tag
):
os
.
environ
[
"GEM_PATH"
]
=
"$${fluentd-service:path}/lib/ruby/gems/1.8/"
fluent_service
=
os
.
environ
.
get
(
'FLUENT_SERVICE'
)
os
.
environ
[
"GEM_PATH"
]
=
fluent_service
+
"/lib/ruby/gems/1.8/"
fluentd_cat_exec_comand
=
'$${fluentd-service:path}/bin/fluent-cat --none '
+
tag
fluentd_cat_exec_comand
=
fluent_service
+
'/bin/fluent-cat --none '
+
tag
+
" -p 5438 "
os
.
system
(
"echo + "
+
test_msg
+
" | "
+
fluentd_cat_exec_comand
)
os
.
system
(
"echo + "
+
test_msg
+
" | "
+
fluentd_cat_exec_comand
)
print
(
"Fluent-cat path"
)
print
(
fluentd_cat_exec_comand
)
def
kill_caddy
(
caddy_pid
):
def
kill_caddy
(
caddy_pid
):
print
(
"caddy pid = "
)
os
.
system
(
"kill -TSTP %s"
%
caddy_pid
)
print
(
caddy_pid
)
print
(
"Caddy is stopped."
)
kill_caddy_cmd
=
"kill -TSTP "
+
caddy_pid
os
.
system
(
kill_caddy_cmd
)
print
(
"Caddy is killed"
)
def
start_caddy
(
caddy_pid
):
def
start_caddy
(
caddy_pid
):
start_caddy_cmd
=
"kill -CONT "
+
caddy_pid
os
.
system
(
"kill -CONT %s"
%
caddy_pid
)
os
.
system
(
start_caddy_cmd
)
print
(
"Caddy is restarted."
)
print
(
"Caddy is restarted"
)
def
find_tag
(
s
,
start
,
end
):
def
find_tag
(
s
,
start
,
end
):
return
(
s
.
split
(
start
))[
1
].
split
(
end
)[
0
]
return
(
s
.
split
(
start
))[
1
].
split
(
end
)[
0
]
def
main
():
port
=
9443
server_address
=
(
'0.0.0.0'
,
port
)
httpd
=
HTTPServer
(
server_address
,
TestServerHandler
)
thread
=
threading
.
Thread
(
target
=
httpd
.
serve_forever
)
thread
.
start
()
print
'Starting http...'
time
.
sleep
(
15
)
stream
=
StringIO
()
runner
=
unittest
.
TextTestRunner
(
verbosity
=
2
,
stream
=
stream
)
result
=
runner
.
run
(
unittest
.
makeSuite
(
TestPost
))
print
'Tests run '
,
result
.
testsRun
print
'Errors '
,
result
.
errors
print
"Failures "
,
result
.
failures
stream
.
seek
(
0
)
print
'Test output
\
n
'
,
stream
.
read
()
time
.
sleep
(
30
)
httpd
.
shutdown
()
print
(
posted_data
)
print
(
"all posted data : "
)
print
(
all_data
)
return
result
.
testsRun
,
result
.
errors
,
result
.
failures
,
stream
.
getvalue
()
if
__name__
==
"__main__"
:
main
()
\ No newline at end of file
software/erp5testnode/testsuite/fluentTest/tests/utils.py
0 → 100644
View file @
71b5141f
##############################################################################
#
# Copyright (c) 2018 Nexedi SA and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
import
unittest
import
os
import
socket
from
contextlib
import
closing
import
logging
from
erp5.util.testnode.SlapOSControler
import
SlapOSControler
from
erp5.util.testnode.ProcessManager
import
ProcessManager
import
slapos
def
findFreeTCPPort
(
ip
=
''
):
"""Find a free TCP port to listen to.
"""
family
=
socket
.
AF_INET6
if
':'
in
ip
else
socket
.
AF_INET
with
closing
(
socket
.
socket
(
family
,
socket
.
SOCK_STREAM
))
as
s
:
s
.
bind
((
ip
,
0
))
return
s
.
getsockname
()[
1
]
class
SlapOSInstanceTestCase
(
unittest
.
TestCase
):
@
classmethod
def
getSoftwareURLList
(
cls
):
"""Return URL of software releases to install.
To be defined by subclasses.
"""
raise
NotImplementedError
()
@
classmethod
def
getInstanceParameterDict
(
cls
):
"""Return instance parameters
To be defined by subclasses if they need to request instance with specific
parameters.
"""
return
{}
# TODO: allow subclasses to request a specific software type ?
@
classmethod
def
setUpClass
(
cls
):
try
:
cls
.
_setUpClass
()
except
:
cls
.
tearDownClass
()
raise
@
classmethod
def
_setUpClass
(
cls
):
working_directory
=
os
.
environ
.
get
(
'SLAPOS_TEST_WORKING_DIR'
,
os
.
path
.
join
(
os
.
path
.
dirname
(
__file__
),
'.slapos'
))
# To prevent error: Cannot open an HTTP server: socket.error reported
# AF_UNIX path too long This `working_directory` should not be too deep.
# Socket path is 108 char max on linux
# https://github.com/torvalds/linux/blob/3848ec5/net/unix/af_unix.c#L234-L238
if
len
(
working_directory
+
'/inst/supervisord.socket.xxxxxxx'
)
>
108
:
raise
RuntimeError
(
'working directory ( {} ) is too deep, try setting '
'SLAPOS_TEST_WORKING_DIR'
.
format
(
working_directory
))
if
not
os
.
path
.
exists
(
working_directory
):
os
.
mkdir
(
working_directory
)
cls
.
config
=
config
=
{
"working_directory"
:
working_directory
,
"slapos_directory"
:
working_directory
,
"log_directory"
:
working_directory
,
"computer_id"
:
'slapos.test'
,
# XXX
'proxy_database'
:
os
.
path
.
join
(
working_directory
,
'proxy.db'
),
'partition_reference'
:
cls
.
__name__
,
# "proper" slapos command must be in $PATH
'slapos_binary'
:
'slapos'
,
}
# Some tests are expecting that local IP is not set to 127.0.0.1
ipv4_address
=
os
.
environ
.
get
(
'LOCAL_IPV4'
,
'127.0.1.1'
)
ipv6_address
=
os
.
environ
[
'GLOBAL_IPV6'
]
config
[
'proxy_host'
]
=
config
[
'ipv4_address'
]
=
ipv4_address
config
[
'ipv6_address'
]
=
ipv6_address
config
[
'proxy_port'
]
=
findFreeTCPPort
(
ipv4_address
)
config
[
'master_url'
]
=
'http://{proxy_host}:{proxy_port}'
.
format
(
**
config
)
cls
.
_process_manager
=
process_manager
=
ProcessManager
()
# XXX this code is copied from testnode code
slapos_controler
=
SlapOSControler
(
working_directory
,
config
)
slapproxy_log
=
os
.
path
.
join
(
config
[
'log_directory'
],
'slapproxy.log'
)
logger
=
logging
.
getLogger
(
__name__
)
logger
.
debug
(
'Configured slapproxy log to %r'
,
slapproxy_log
)
software_url_list
=
cls
.
getSoftwareURLList
()
slapos_controler
.
initializeSlapOSControler
(
slapproxy_log
=
slapproxy_log
,
process_manager
=
process_manager
,
reset_software
=
False
,
software_path_list
=
software_url_list
)
process_manager
.
supervisord_pid_file
=
os
.
path
.
join
(
slapos_controler
.
instance_root
,
'var'
,
'run'
,
'supervisord.pid'
)
software_status_dict
=
slapos_controler
.
runSoftwareRelease
(
config
,
environment
=
os
.
environ
)
# TODO: log more details in this case
assert
software_status_dict
[
'status_code'
]
==
0
instance_parameter_dict
=
cls
.
getInstanceParameterDict
()
instance_status_dict
=
slapos_controler
.
runComputerPartition
(
config
,
cluster_configuration
=
instance_parameter_dict
,
environment
=
os
.
environ
)
# TODO: log more details in this case
assert
instance_status_dict
[
'status_code'
]
==
0
# FIXME: similar to test node, only one (root) partition is really supported for now.
computer_partition_list
=
[]
for
i
in
range
(
len
(
software_url_list
)):
computer_partition_list
.
append
(
slapos_controler
.
slap
.
registerOpenOrder
().
request
(
software_url_list
[
i
],
# This is how testnode's SlapOSControler name created partitions
partition_reference
=
'testing partition {i}'
.
format
(
i
=
i
,
**
config
),
partition_parameter_kw
=
instance_parameter_dict
))
# expose some class attributes so that tests can use them:
# the ComputerPartition instances, to getInstanceParmeterDict
cls
.
computer_partition
=
computer_partition_list
[
0
]
# the path of the instance on the filesystem, for low level inspection
cls
.
computer_partition_root_path
=
os
.
path
.
join
(
config
[
'working_directory'
],
'inst'
,
cls
.
computer_partition
.
getId
())
@
classmethod
def
tearDownClass
(
cls
):
if
hasattr
(
cls
,
'_process_manager'
):
cls
.
_process_manager
.
killPreviousRun
()
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment