Merge pull request #5 from apache/dev (6e1dbd89) · Commits · SUMMER2020 / students / proj-2002251

.github/workflows/ci_e2e.yml

0 → 100644

+67 −0

Original line number	Diff line number	Diff line
		#
		# Licensed to the Apache Software Foundation (ASF) under one or more
		# contributor license agreements. See the NOTICE file distributed with
		# this work for additional information regarding copyright ownership.
		# The ASF licenses this file to You under the Apache License, Version 2.0
		# (the "License"); you may not use this file except in compliance with
		# the License. You may obtain a copy of the License at
		#
		# http://www.apache.org/licenses/LICENSE-2.0
		#
		# Unless required by applicable law or agreed to in writing, software
		# distributed under the License is distributed on an "AS IS" BASIS,
		# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
		# See the License for the specific language governing permissions and
		# limitations under the License.
		#

		on: ["push", "pull_request"]
		env:
		DOCKER_DIR: ./docker
		LOG_DIR: /tmp/dolphinscheduler

		name: e2e Test

		jobs:

		build:
		name: Test
		runs-on: ubuntu-latest
		steps:

		- uses: actions/checkout@v1
		with:
		submodules: true
		- uses: actions/cache@v1
		with:
		path: ~/.m2/repository
		key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
		restore-keys: \|
		${{ runner.os }}-maven-
		- name: Build Image
		run: \|
		export VERSION=`cat $(pwd)/pom.xml\| grep "SNAPSHOT</version>" \| awk -F "-SNAPSHOT" '{print $1}' \| awk -F ">" '{print $2}'`
		sh ./dockerfile/hooks/build
		- name: Docker Run
		run: \|
		VERSION=`cat $(pwd)/pom.xml\| grep "SNAPSHOT</version>" \| awk -F "-SNAPSHOT" '{print $1}' \| awk -F ">" '{print $2}'`
		docker run -dit -e POSTGRESQL_USERNAME=test -e POSTGRESQL_PASSWORD=test -p 8888:8888 dolphinscheduler:$VERSION all
		- name: Check Server Status
		run: sh ./dockerfile/hooks/check
		- name: Prepare e2e env
		run: \|
		sudo apt-get install -y libxss1 libappindicator1 libindicator7 xvfb unzip
		wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
		sudo dpkg -i google-chrome*.deb
		sudo apt-get install -f -y
		wget -N https://chromedriver.storage.googleapis.com/80.0.3987.106/chromedriver_linux64.zip
		unzip chromedriver_linux64.zip
		sudo mv -f chromedriver /usr/local/share/chromedriver
		sudo ln -s /usr/local/share/chromedriver /usr/local/bin/chromedriver
		- name: Run e2e Test
		run: cd ./e2e && mvn -B clean test
		- name: Collect logs
		run: \|
		mkdir -p ${LOG_DIR}
		docker logs dolphinscheduler > ${LOG_DIR}/dolphinscheduler.txt
		continue-on-error: true

.github/workflows/ci_ut.yml

+1 −1

Original line number	Diff line number	Diff line
		@@ -20,7 +20,7 @@ env:
		DOCKER_DIR: ./docker
		LOG_DIR: /tmp/dolphinscheduler

		name: Test Coveralls Parallel
		name: Unit Test

		jobs:

README.md

+9 −17

Original line number	Diff line number	Diff line
		@@ -45,17 +45,16 @@ HA is supported by itself \| All process definition operations are visualized, dr
		Overload processing: Task queue mechanism, the number of schedulable tasks on a single machine can be flexibly configured, when too many tasks will be cached in the task queue, will not cause machine jam. \| One-click deployment \| Supports traditional shell tasks, and also support big data platform task scheduling: MR, Spark, SQL (mysql, postgresql, hive, sparksql), Python, Procedure, Sub_Process \| \|




		### System partial screenshot

		![image](https://user-images.githubusercontent.com/48329107/61368744-1f5f3b00-a8c1-11e9-9cf1-10f8557a6b3b.png)

		![image](https://user-images.githubusercontent.com/48329107/61368966-9dbbdd00-a8c1-11e9-8dcc-a9469d33583e.png)

		![image](https://user-images.githubusercontent.com/48329107/61372146-f347b800-a8c8-11e9-8882-66e8934ada23.png)


		![home page](https://user-images.githubusercontent.com/15833811/75218288-bf286400-57d4-11ea-8263-d639c6511d5f.jpg)
		![dag](https://user-images.githubusercontent.com/15833811/75236750-3374fe80-57f9-11ea-857d-62a66a5a559d.png)
		![process definition list page](https://user-images.githubusercontent.com/15833811/75216886-6f479e00-57d0-11ea-92dd-66e7640a186f.png)
		![view task log online](https://user-images.githubusercontent.com/15833811/75216924-9900c500-57d0-11ea-91dc-3522a76bdbbe.png)
		![resource management](https://user-images.githubusercontent.com/15833811/75216984-be8dce80-57d0-11ea-840d-58546edc8788.png)
		![monitor](https://user-images.githubusercontent.com/59273635/75625839-c698a480-5bfc-11ea-8bbe-895b561b337f.png)
		![security](https://user-images.githubusercontent.com/15833811/75236441-bfd2f180-57f8-11ea-88bd-f24311e01b7e.png)
		![treeview](https://user-images.githubusercontent.com/15833811/75217191-3fe56100-57d1-11ea-8856-f19180d9a879.png)
		### Document

		- <a href="https://dolphinscheduler.apache.org/en-us/docs/1.2.0/user_doc/backend-deployment.html" target="_blank">Backend deployment documentation</a>
		@@ -100,16 +99,9 @@ It is because of the shoulders of these open source projects that the birth of t
		### Get Help
		1. Submit an issue
		1. Subscribe the mail list : https://dolphinscheduler.apache.org/en-us/docs/development/subscribe.html. then send mail to dev@dolphinscheduler.apache.org
		1. Contact WeChat group manager, ID 510570367. This is for Mandarin(CN) discussion.
		1. Contact WeChat(dailidong66). This is just for Mandarin(CN) discussion.

		### License
		Please refer to [LICENSE](https://github.com/apache/incubator-dolphinscheduler/blob/dev/LICENSE) file.

README_zh_CN.md

+11 −3

Original line number	Diff line number	Diff line
		@@ -36,11 +36,19 @@ Dolphin Scheduler Official Website

		### 系统部分截图

		![](http://geek.analysys.cn/static/upload/221/2019-03-29/0a9dea80-fb02-4fa5-a812-633b67035ffc.jpeg)
		![home page](https://user-images.githubusercontent.com/15833811/75208819-abbad000-57b7-11ea-8d3c-67e7c270671f.jpg)

		![](http://geek.analysys.cn/static/upload/221/2019-04-01/83686def-a54f-4169-8cae-77b1f8300cc1.png)
		![dag](https://user-images.githubusercontent.com/15833811/75209584-93e44b80-57b9-11ea-952e-537fb24ec72d.jpg)

		![](http://geek.analysys.cn/static/upload/221/2019-03-29/83c937c7-1793-4d7a-aa28-b98460329fe0.jpeg)
		![log](https://user-images.githubusercontent.com/15833811/75209645-c55d1700-57b9-11ea-94d4-e3fa91ab5218.jpg)

		![gantt](https://user-images.githubusercontent.com/15833811/75209640-c0986300-57b9-11ea-878e-a2098533ad44.jpg)

		![resources](https://user-images.githubusercontent.com/15833811/75209403-11f42280-57b9-11ea-9b59-d4be77063553.jpg)

		![monitor](https://user-images.githubusercontent.com/15833811/75209631-b5ddce00-57b9-11ea-8d22-cdf15cf0ee25.jpg)

		![security](https://user-images.githubusercontent.com/15833811/75209633-baa28200-57b9-11ea-9def-94bef2e212a7.jpg)

		### 文档

ambari_plugin/common-services/DOLPHIN/1.2.1/alerts.json

0 → 100644

+158 −0

Original line number	Diff line number	Diff line
		{
		"DOLPHIN": {
		"service": [],
		"DOLPHIN_API": [
		{
		"name": "dolphin_api_port_check",
		"label": "dolphin_api_port_check",
		"description": "dolphin_api_port_check.",
		"interval": 10,
		"scope": "ANY",
		"source": {
		"type": "PORT",
		"uri": "{{dolphin-application-api/server.port}}",
		"default_port": 12345,
		"reporting": {
		"ok": {
		"text": "TCP OK - {0:.3f}s response on port {1}"
		},
		"warning": {
		"text": "TCP OK - {0:.3f}s response on port {1}",
		"value": 1.5
		},
		"critical": {
		"text": "Connection failed: {0} to {1}:{2}",
		"value": 5.0
		}
		}
		}
		}
		],
		"DOLPHIN_LOGGER": [
		{
		"name": "dolphin_logger_port_check",
		"label": "dolphin_logger_port_check",
		"description": "dolphin_logger_port_check.",
		"interval": 10,
		"scope": "ANY",
		"source": {
		"type": "PORT",
		"uri": "{{dolphin-common/loggerserver.rpc.port}}",
		"default_port": 50051,
		"reporting": {
		"ok": {
		"text": "TCP OK - {0:.3f}s response on port {1}"
		},
		"warning": {
		"text": "TCP OK - {0:.3f}s response on port {1}",
		"value": 1.5
		},
		"critical": {
		"text": "Connection failed: {0} to {1}:{2}",
		"value": 5.0
		}
		}
		}
		}
		],
		"DOLPHIN_MASTER": [
		{
		"name": "DOLPHIN_MASTER_CHECK",
		"label": "check dolphin scheduler master status",
		"description": "",
		"interval":10,
		"scope": "HOST",
		"enabled": true,
		"source": {
		"type": "SCRIPT",
		"path": "DOLPHIN/1.2.1/package/alerts/alert_dolphin_scheduler_status.py",
		"parameters": [

		{
		"name": "connection.timeout",
		"display_name": "Connection Timeout",
		"value": 5.0,
		"type": "NUMERIC",
		"description": "The maximum time before this alert is considered to be CRITICAL",
		"units": "seconds",
		"threshold": "CRITICAL"
		},
		{
		"name": "alertName",
		"display_name": "alertName",
		"value": "DOLPHIN_MASTER",
		"type": "STRING",
		"description": "alert name"
		}
		]
		}
		}
		],
		"DOLPHIN_WORKER": [
		{
		"name": "DOLPHIN_WORKER_CHECK",
		"label": "check dolphin scheduler worker status",
		"description": "",
		"interval":10,
		"scope": "HOST",
		"enabled": true,
		"source": {
		"type": "SCRIPT",
		"path": "DOLPHIN/1.2.1/package/alerts/alert_dolphin_scheduler_status.py",
		"parameters": [

		{
		"name": "connection.timeout",
		"display_name": "Connection Timeout",
		"value": 5.0,
		"type": "NUMERIC",
		"description": "The maximum time before this alert is considered to be CRITICAL",
		"units": "seconds",
		"threshold": "CRITICAL"
		},
		{
		"name": "alertName",
		"display_name": "alertName",
		"value": "DOLPHIN_WORKER",
		"type": "STRING",
		"description": "alert name"
		}
		]
		}
		}
		],
		"DOLPHIN_ALERT": [
		{
		"name": "DOLPHIN_DOLPHIN_ALERT_CHECK",
		"label": "check dolphin scheduler alert status",
		"description": "",
		"interval":10,
		"scope": "HOST",
		"enabled": true,
		"source": {
		"type": "SCRIPT",
		"path": "DOLPHIN/1.2.1/package/alerts/alert_dolphin_scheduler_status.py",
		"parameters": [

		{
		"name": "connection.timeout",
		"display_name": "Connection Timeout",
		"value": 5.0,
		"type": "NUMERIC",
		"description": "The maximum time before this alert is considered to be CRITICAL",
		"units": "seconds",
		"threshold": "CRITICAL"
		},
		{
		"name": "alertName",
		"display_name": "alertName",
		"value": "DOLPHIN_ALERT",
		"type": "STRING",
		"description": "alert name"
		}
		]
		}
		}
		]
		}
		}
		No newline at end of file