marker/shuffle-cracked
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #1 from dhaval055/patch-1

Browse Source 
Patch 1
This commit is contained in:
Dhaval Dave
2022-05-02 18:43:46 +05:30
committed by GitHub
parent 87e8a5d549 941630911e
commit de970341e7
223 changed files with 72946 additions and 61495 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.dockerignore
-15
View File
@@ -1,15 +0,0 @@
**/.git*
**/.gitlab-ci.yml
**/.env
**/.dockerignore
**/Dockerfile*
**/node_modules
buildSrc/libs
.gradle/
build/
**build/
.idea/
!.idea/codeStyles/codeStyleConfig.xml
.DS_Store
*.log
out/
.env
+36 -5
View File
@@ -2,16 +2,18 @@
ORG_ID=Shuffle
ENVIRONMENT_NAME=Shuffle
# Remote github config for first load
SHUFFLE_DOWNLOAD_WORKFLOW_LOCATION=
SHUFFLE_DOWNLOAD_WORKFLOW_USERNAME=
SHUFFLE_DOWNLOAD_WORKFLOW_PASSWORD=
SHUFFLE_DOWNLOAD_WORKFLOW_BRANCH=
SHUFFLE_APP_DOWNLOAD_LOCATION=https://github.com/frikky/shuffle-apps
SHUFFLE_APP_DOWNLOAD_LOCATION=https://github.com/shuffle/python-apps
SHUFFLE_DOWNLOAD_AUTH_USERNAME=
SHUFFLE_DOWNLOAD_AUTH_PASSWORD=
SHUFFLE_DOWNLOAD_AUTH_BRANCH=
SHUFFLE_APP_FORCE_UPDATE=false
# User config for first load. Username & PW: min length 3
SHUFFLE_DEFAULT_USERNAME=
@@ -20,26 +22,55 @@ SHUFFLE_DEFAULT_APIKEY=
# Local location of your app directory. Can't use ~/
# Files will get better at some point. Right now: local saving.
SHUFFLE_APP_HOTLOAD_FOLDER=./shuffle-apps
SHUFFLE_APP_HOTLOAD_LOCATION=./shuffle-apps
SHUFFLE_FILE_LOCATION=./shuffle-files
# Encryption modifier. This HAS to be set to encrypt any authentication being used in Shuffle. This is put together with other relevant values to ensure multiple parts are needed to decrypt.
# If this key is lost or changed, you will have to reauthenticate all apps.
SHUFFLE_ENCRYPTION_MODIFIER=
# Other configs
BASE_URL=http://shuffle-backend:5001
BACKEND_HOSTNAME=shuffle-backend
BACKEND_PORT=5001
FRONTEND_PORT=3001
FRONTEND_PORT_HTTPS=3443
# CHANGE THIS IF YOU WANT GOOD LOCAL EXECUTIONS:
OUTER_HOSTNAME=shuffle-backend
DB_LOCATION=./shuffle-database
DOCKER_API_VERSION=1.40
# Proxy configurations. SHUFFLE_PASS_WORKER_PROXY must be FALSE to not pass the proxy information to sub-apps.
# PS: It will skip proxy for
SHUFFLE_HTTP_PROXY=
SHUFFLE_HTTPS_PROXY=
HTTP_PROXY=
HTTPS_PROXY=
SHUFFLE_PASS_WORKER_PROXY=TRUE
SHUFFLE_PASS_APP_PROXY=FALSE
TZ=Europe/Amsterdam # Timezone-handler in Orborus, Worker and Apps
ORBORUS_CONTAINER_NAME= # Used to FIND the containername. cgroup v2: issue 501
SHUFFLE_BASE_IMAGE_REGISTRY=ghcr.io
SHUFFLE_BASE_IMAGE_NAME=frikky
SHUFFLE_BASE_IMAGE_TAG_SUFFIX="-0.8.3"
SHUFFLE_BASE_IMAGE_REGISTRY=ghcr.io
SHUFFLE_BASE_IMAGE_TAG_SUFFIX="-0.8.80"
# Used for auto-cleanup of containers. REALLY important at scale.
SHUFFLE_CONTAINER_AUTO_CLEANUP=false
SHUFFLE_ELASTIC=true
SHUFFLE_LOGS_DISABLED=false
SHUFFLE_CHAT_DISABLED=false
SHUFFLE_RERUN_SCHEDULE=300
# DATABASE CONFIGURATIONS
DATASTORE_EMULATOR_HOST=shuffle-database:8000
#SHUFFLE_OPENSEARCH_URL=http://shuffle-opensearch:9200
SHUFFLE_OPENSEARCH_URL=https://shuffle-opensearch:9200
SHUFFLE_OPENSEARCH_USERNAME=admin
SHUFFLE_OPENSEARCH_PASSWORD=admin
SHUFFLE_OPENSEARCH_CERTIFICATE_FILE=
SHUFFLE_OPENSEARCH_APIKEY=
SHUFFLE_OPENSEARCH_CLOUDID=
SHUFFLE_OPENSEARCH_PROXY=
SHUFFLE_OPENSEARCH_INDEX_PREFIX=
SHUFFLE_OPENSEARCH_SKIPSSL_VERIFY=true
.github/CONTRIBUTING.md
+48
View File
@@ -0,0 +1,48 @@
# Contributing to Shuffle
First off, thank you for contributing to [Shuffle](https://shuffler.io/)! Your talents and your contributions are greatly appreciated. With Shuffle, we aim to make cybersecurity more accessible, and keep that in mind with everything we make.
## Opening a new issue
If you find a bug or think of an improvement or fix, please open a [new issue](https://github.com/frikky/Shuffle/issues/new). Outline every step necessary to reproduce the bug. Include screenshots, logs and/or code examples where applicable. The more thorough you are, the better.
## What you can work on
There are a lot of things to work on in our complex ecosystem. The most pressing issues are documentation, use-cases and content-creation, but any help is appreciated. We'll make sure you get the help you need to get started. Below is an incomplete list of items. If you see an issue, tell us or fix it! :)
#### App Creation (Python & GUI w/OpenAPI)
As with everything else, app creation for Shuffle is made as accessibl as possible with the app editor. However, there are some instances where it can't do the job, and you'll have to write Python code. The App Editor generates OpenAPI specifications and can be widely shared, while Python apps only work for Shuffle and NSA's WALKOFF (which Shuffle is based on). You can find our [OpenAPI apps here](https://github.com/frikky/security-openapis) and our [Python apps here](https://github.com/frikky/shuffle-apps). Apps in these repositories are automatically available after installation. Shuffle apps are searchable on [https://shuffler.io](https://shuffler.io/search).
#### Workflow creation (GUI & Conceptualizing)
Workflows are where the magic of Shuffle automation happens. Our current ones [are outlined here](https://github.com/frikky/security-openapis), and will be automatically imported into Shuffle instances in the future. They are split into Prepare and Response, but don't necessarily have to be. If you'd like to talk about workflow creation or use-cases in general, either Open a [new issue](https://github.com/frikky/shuffle-workflows/issues/new) or send us an email at [frikky@shuffler.io](mailto:frikky@shuffler.io)
#### Documentation (Markdown)
Documentation is essential to any product, and Shuffle is no exception. Documentation in Shuffle uses markdown and is located in the [shuffle-docs](https://github.com/frikky/shuffle-docs/tree/master/docs) repository. These are then loaded into Shuffle when someone visits [https://shuffler/docs/about](https://shuffler/docs/about), then cached for later use. If you make an edit, expect it on our website in about an hour.
#### Frontend (ReactJS)
The frontend of Shuffle is what everyone sees when they log in. Our goal here is to make it easy to get started and keep going with Shuffle - removing any blockers from the point of accessibility. If you'd like to get started, find [an issue](https://github.com/frikky/Shuffle/issues) and check the [installation guide](https://github.com/frikky/Shuffle/blob/master/install-guide.md#local-development-installation) for setting it up locally without Docker.
#### Backend (Golang)
The backend of Shuffle is our REST API Server that runs in the background, handling all the API-calls in general, whether from users or apps. If you'd like to get started, find [an issue](https://github.com/frikky/Shuffle/issues) and check the [installation guide](https://github.com/frikky/Shuffle/blob/master/install-guide.md#local-development-installation) for setting it up locally without Docker.
#### Scaling (Golang & Python)
Shuffle runs using Docker, and is built to scale. There are many areas that may revolve around scaling, but the main issues come down to how we use Docker in our [architecture](https://shuffler.io/docs/architecture). If you want to help by submitting Helm charts (K8s), Docker swarm configurations, blogposts, or talk about code changes that would help scaling - please reach out (or just start building!), and we can discuss the possibilities. Make sure to read about the architecture first :)
#### Testing
Whether it's security testing, code testing or CI/CD, we could always need another hand. E.g. an example of CI/CD used for apps can be found [here](https://github.com/Shuffle/Shuffle-apps/blob/master/.github/workflows/ci.yaml), but we don't at all limit the scope to Github actions. If you find a security issue, whether open source or not, please contact [security@shuffler.io](mailto:security@shuffler.io) or [contact us on our website](https://shuffler.io/contact).
#### Community
What is a product without a community? Want to help out? Whether it be through blogposts, videos or community management, don't hesitate to [reach out](https://shuffler.io/contact) if you would like to help, and get a more keen understanding of how we work. (PS: We're hiring)
## Working on an issue
**Shuffle** uses the [GitHub flow](https://guides.github.com/introduction/flow/index.html). All project changes are made through pull requests.
If you see an issue that you would like to work on, leave a quick comment or just get cracking.
### License
All contributions are made under either the **GNU Affero General Public License v3.0** or **MIT** license. See below for further details.
* [Main project license - AGPLv3](https://github.com/frikky/Shuffle/blob/master/LICENSE)
* [Apps - MIT](https://github.com/frikky/Shuffle-apps/blob/master/LICENSE)
* [Workflows - MIT](https://github.com/frikky/Shuffle-workflows/blob/master/LICENSE)
* [Documentation - MIT](https://github.com/frikky/Shuffle-docs/blob/master/LICENSE)
.github/install-guide.html
+126
View File
File diff suppressed because one or more lines are too long
.github/install-guide.md
+127
View File
@@ -0,0 +1,127 @@
# Installation guide
Installation of Shuffle is currently only available in docker. Looking for how to update Shuffle? Check the [updating guide](https://shuffler.io/docs/configuration#updating_shuffle)
This document outlines a an introduction environment which is not scalable. [Read here](https://shuffler.io/docs/configuration#production_readiness) for information on production readiness. This also includes system requirements and configurations for Swarm or K8s.
# Docker - *nix
The Docker setup is done with docker-compose
**PS: if you're setting up Shuffle on Windows, go to the next step (Windows Docker setup)**
1. Make sure you have [Docker](https://docs.docker.com/get-docker/) and [docker-compose](https://docs.docker.com/compose/install/) installed.
2. Download Shuffle
```bash
git clone https://github.com/frikky/Shuffle
cd Shuffle
```
3. Fix prerequisites for the Opensearch database (Elasticsearch):
```bash
mkdir shuffle-database
sudo chown -R 1000:1000 shuffle-database
```
4. Run docker-compose.
```bash
docker-compose up -d
```
When you're done, skip to the "After installation" step below.
## Windows with WSL
This step is for setting up with Docker on windows from scratch.
1. Make sure you have [Docker](https://docs.docker.com/docker-for-windows/install/) and [docker-compose](https://docs.docker.com/compose/install/) installed. WSL2 may be required.
2. Go to https://github.com/frikky/shuffle/releases and download the latest .zip release (or install git)
3. Unzip the folder and enter it
4. Open the .env file and change the line with "OUTER_HOSTNAME" to contain your IP:
```bash
OUTER_HOSTNAME=YOUR.IP.HERE
```
6. Run docker-compose
```bash
docker-compose up -d
```
### Configurations (proxies, default users etc.)
https://shuffler.io/docs/configuration
### After installation
1. After installation, go to http://localhost:3001 (or your servername - https is on port 3443)
2. Now set up your admin account (username & password). Shuffle doesn't have a default username and password.
3. Sign in with the same Username & Password! Go to /apps and see if you have any apps yet. If not - you may need to [configure proxies](https://shuffler.io/docs/configuration#production_readiness)
4. Check out https://shuffler.io/docs/configuration as it has a lot of useful information to get started
![Admin account setup](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_adminaccount.png)
### Useful info
* Check out [getting started](https://shuffler.io/docs/getting_started)
* The default state of Shuffle is NOT scalable. See [production setup](https://shuffler.io/docs/configuration#production_readiness) for more info
* The server is available on http://localhost:3001 (or your servername)
* Further configurations can be done in docker-compose.yml and .env.
* Default database location is in the same folder: ./shuffle-database
# Local development installation
Local development is pretty straight forward with **ReactJS** and **Golang**. This part is intended to help you run the code for development purposes. We recommend having Shuffle running with the Docker-compose, then manually running the portion that you want to test and/or edit.
**PS: You have to stop the Backend Docker container to get this one working**
**PPS: Use the "Launch" branch when developing to get it set up easier**
## Frontend - ReactJS /w cytoscape
http://localhost:3000 - Requires [npm](https://nodejs.org/en/download/)/[yarn](https://yarnpkg.com/lang/en/docs/install/#debian-stable)/your preferred manager. Runs independently from backend.
```bash
cd frontend
npm i
npm start
```
## Backend - Golang
http://localhost:5001 - REST API - requires [>=go1.13](https://golang.org/dl/)
```bash
export SHUFFLE_OPENSEARCH_URL="http://localhost:9200"
export SHUFFLE_ELASTIC=true
cd backend/go-app
go run main.go walkoff.go docker.go
```
Large portions of the backend is written in another repository - [shuffle-shared](https://github.com/frikky/shuffle-shared). If you want to update any of this code and test in realtime, we recommend following these steps:
1. Clone shuffle-shared to a local repository
2. Open the Shuffle backend's go.mod file (./shuffle/backend/go.mod) (**NOT** in shuffle-shared)
3. Change the following line to point to your directory AFTER the =>
```
//replace github.com/frikky/shuffle-shared => ../../../../git/shuffle-shared
```
4. Make the changes you want, then restart the backend server!
5. With your changes made, make a pull request :fire:
**WINDOWS USERS:** You'll have to to add the "export" part as an environment variable.
## Database - Opensearch
Make sure this is running through the docker-compose, and that the backend points to it with SHUFFLE_OPENSEARCH_URL defined
## Orborus
Execution of Workflows:
PS: This requires some specific environment variables
```
cd functions/onprem/orborus
go run orborus.go
```
Environments (modify for Windows):
```
export ORG_ID=Shuffle
export ENVIRONMENT_NAME=Shuffle
export BASE_URL=http://YOUR-IP:5001
export DOCKER_API_VERSION=1.40
```
**WINDOWS USERS:** You'll have to to add the "export" part as an environment variable.
AND THAT's it - hopefully it worked. If it didn't please email [frikky@shuffler.io](mailto:frikky@shuffler.io)
.github/workflows/ci.yaml
+66
View File
@@ -0,0 +1,66 @@
name: ci
on:
push:
branches: master
jobs:
main:
runs-on: ubuntu-latest
continue-on-error: ${{ matrix.experimental }}
strategy:
fail-fast: false
matrix:
include:
- app: frontend
path: frontend
version: 0.8.3
experimental: true
- app: backend
path: backend
version: 0.8.3
experimental: false
- app: orborus
path: functions/onprem/orborus
version: 0.8.0
experimental: false
- app: database
path: backend/database
version: 0.8.0
experimental: false
steps:
-
name: Checkout
uses: actions/checkout@v2
-
name: Set up QEMU
uses: docker/setup-qemu-action@v1
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
-
name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Use below configuration for ghcr.io
# with:
# registry: ghcr.io
# username: ${{ github.repository_owner }}
# password: ${{ secrets.CR_PAT }}
-
name: Build and push
id: docker_build
uses: docker/build-push-action@v2
env:
BUILDX_NO_DEFAULT_LOAD: true
with:
context: ${{ matrix.path }}/
file: ${{ matrix.path }}/Dockerfile
platforms: linux/amd64,linux/arm64
#,linux/386 - no node image I guess?
push: true
tags: ${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.app }}:${{ matrix.version }}
-
name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
.github/workflows/codeql-analysis.yml
+6 -2
View File
@@ -13,10 +13,14 @@ name: "CodeQL"
on:
push:
branches: [ master ]
branches:
- master
- launch
pull_request:
# The branches below must be a subset of the branches above
branches: [ master ]
branches:
- master
- launch
schedule:
- cron: '38 16 * * 4'
.github/workflows/snyk-container-analysis.yml
+47
View File
@@ -0,0 +1,47 @@
# A sample workflow which checks out the code, builds a container
# image using Docker and scans that image for vulnerabilities using
# Snyk. The results are then uploaded to GitHub Security Code Scanning
#
# For more examples, including how to limit scans to only high-severity
# issues, monitor images for newly disclosed vulnerabilities in Snyk and
# fail PR checks for new vulnerabilities, see https://github.com/snyk/actions/
name: Snyk Container
on:
push:
branches:
- master
- launch
pull_request:
# The branches below must be a subset of the branches above
branches:
- master
- launch
schedule:
- cron: '18 4 * * 3'
jobs:
snyk:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Build a Docker image
run: docker build -t your/image-to-test .
- name: Run Snyk to check Docker image for vulnerabilities
# Snyk can be used to break the build when it detects vulnerabilities.
# In this case we want to upload the issues to GitHub Code Scanning
continue-on-error: true
uses: snyk/actions/docker@master
env:
# In order to use the Snyk Action you will need to have a Snyk API token.
# More details in https://github.com/snyk/actions#getting-your-snyk-token
# or you can signup for free at https://snyk.io/login
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
image: your/image-to-test
args: --file=Dockerfile
- name: Upload result to GitHub Code Scanning
uses: github/codeql-action/upload-sarif@v1
with:
sarif_file: snyk.sarif
.github/workflows/snyk-infrastructure-analysis.yml
+46
View File
@@ -0,0 +1,46 @@
# A sample workflow which checks out your Infrastructure as Code Configuration files,
# such as Kubernetes, Helm & Terraform and scans them for any security issues.
# The results are then uploaded to GitHub Security Code Scanning
#
# For more examples, including how to limit scans to only high-severity issues
# and fail PR checks, see https://github.com/snyk/actions/
name: Snyk Infrastructure as Code
on:
push:
branches:
- master
- launch
pull_request:
# The branches below must be a subset of the branches above
branches:
- master
- launch
schedule:
- cron: '41 16 * * 2'
jobs:
snyk:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Run Snyk to check configuration files for security issues
# Snyk can be used to break the build when it detects security issues.
# In this case we want to upload the issues to GitHub Code Scanning
continue-on-error: true
uses: snyk/actions/iac@master
env:
# In order to use the Snyk Action you will need to have a Snyk API token.
# More details in https://github.com/snyk/actions#getting-your-snyk-token
# or you can signup for free at https://snyk.io/login
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
# Add the path to the configuration file that you would like to test.
# For example `deployment.yaml` for a Kubernetes deployment manifest
# or `main.tf` for a Terraform configuration file
file: your-file-to-test.yaml
- name: Upload result to GitHub Code Scanning
uses: github/codeql-action/upload-sarif@v1
with:
sarif_file: snyk.sarif
.gitignore
+11
View File
@@ -18,3 +18,14 @@ functions/generated_apps
backend/onprem/app_sdk/apps
*test.py
*.exe
*debug*
shuffle-database/batch_metrics_enabled.conf
shuffle-database/logging_enabled.conf
shuffle-database/nodes
shuffle-database/performance_analyzer_enabled.conf
shuffle-database/rca_enabled.conf
*/package-lock.json
README.md
+55 -25
View File
@@ -1,20 +1,37 @@
# Shuffle
[Shuffle](https://shuffler.io) is an automation platform to unify your security services (SOAR). It has thousands of premade integrations and is based on open frameworks like OpenAPI and Mitre Att&ck. The workflow editor is based on a no-code thought process to empower non-developers, and the app creator makes you able to integrate any platform in minutes.
<h1 align="center">
[![Discord](https://img.shields.io/discord/463752820026376202.svg?label=&logo=discord&logoColor=ffffff&color=7389D8&labelColor=6A7EC2)](https://discord.gg/B2CBzUm)
[![Shuffle Logo](https://github.com/frikky/Shuffle/blob/launch/frontend/public/images/Shuffle_logo_new.png)](https://shuffler.io)
![Example Shuffle webhook integration](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_webhook.png)
Shuffle Automation
</h1><h4 align="center">
[Shuffle](https://shuffler.io) is an automation platform for and by the community, focusing on accessibility for anyone to automate. Security operations is complex, but it doesn't have to be.
[_Key Features_](https://shuffler.io/docs/features) —
[_Community & Support_](https://discord.gg/B2CBzUm) —
[_Documentation_](https://shuffler.io/docs) —
[_Getting Started_](https://shuffler.io/docs/getting_started) —
[_Development_](https://github.com/frikky/Shuffle/blob/master/.github/CONTRIBUTING.md)
Follow us on Twitter at [@shuffleio](https://twitter.com/shuffleio).
</h4>
![Example Shuffle webhook integration](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/github_shuffle_img.png)
## Try it
* Self-hosted: Check out the [installation guide](https://github.com/frikky/shuffle/blob/master/install-guide.md)
* Self-hosted: Check out the [installation guide](https://github.com/frikky/shuffle/blob/master/.github/install-guide.md)
* Cloud: Register at https://shuffler.io/register and get cooking (missing a lot of features)
Please consider [sponsoring](https://github.com/sponsors/frikky) the project if you want to see more rapid development.
## Support
* [Discord](https://discord.gg/B2CBzUm)
* [Twitter](https://twitter.com/shuffleio)
* [Email](mailto:frikky@shuffler.io)
* [Open issue](https://github.com/frikky/Shuffle/issues/new)
* [Shuffler.io](https://shuffler.io/contact)
## Blogposts
* [1. Introducing Shuffle](https://medium.com/security-operation-capybara/introducing-shuffle-an-open-source-soar-platform-part-1-58a529de7d12)
@@ -23,25 +40,36 @@ Please consider [sponsoring](https://github.com/sponsors/frikky) the project if
* [4. Real-time executions with TheHive, Cortex and MISP](https://medium.com/@Frikkylikeme/indicators-and-webhooks-with-thehive-cortex-and-misp-open-source-soar-part-4-f70cde942e59)
## Documentation
[Documentation](https://shuffler.io/docs) can be found on https://shuffler.io/docs and is written in https://github.com/frikky/shuffle-docs.
[Documentation](https://shuffler.io/docs) can be found on [https://shuffler.io/docs](https://shuffler.io/docs) and is written here: [https://github.com/frikky/shuffle-docs](https://github.com/frikky/shuffle-docs).
## Related repositories
* Apps: https://github.com/frikky/shuffle-apps
* Workflows: https://github.com/frikky/shuffle-workflows
* Security OpenAPI apps: https://github.com/frikky/security-openapis
* Documentation: https://github.com/frikky/shuffle-docs
* OpenAPI apps: [https://github.com/frikky/security-openapis](https://github.com/frikky/security-openapis)
* Documentation: [https://github.com/frikky/shuffle-docs](https://github.com/frikky/shuffle-docs)
* Workflows: [https://github.com/frikky/shuffle-workflows](https://github.com/frikky/shuffle-workflows)
* Python apps: [https://github.com/frikky/shuffle-apps](https://github.com/frikky/shuffle-apps)
## Features
* Simple workflow automation editor
* Premade apps for a number of security tools
* App creator for [OpenAPI](https://github.com/frikky/OpenAPI-security-definitions)
* Easy to learn Python library for custom apps
## Architecture
![Shuffle Architecture](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_architecture.png)
* Simple, feature rich [workflow editor](https://shuffler.io/docs/workflows)
* App creator using [OpenAPI](https://github.com/frikky/OpenAPI-security-definitions)
* Premade apps for your security tools
* Organization and sub-organization control
* Hybrid resource sharing with shuffler.io (optional)
## Website
https://shuffler.io
[https://shuffler.io](https://shuffler.io)
## Contributing
We want to make the world of cybersecurity more accessible and need all the help we can get. Send an email to [frikky@shuffler](mailto:frikky@shuffler.io) and we'll make sure to give you any training you may need.
These are the main areas to contribute in:
* Frontend (ReactJS)
* Backend (Golang)
* App Creation (Python & GUI w/OpenAPI)
* Documentation (Markdown)
* Workflow creation (GUI & Conceptualizing)
* Content Creation (Blogs, videos etc)
Contributing guidelines are outlined [here](https://github.com/frikky/Shuffle/blob/master/.github/CONTRIBUTING.md).
## Contributors
![ICPL logo](https://github.com/frikky/Shuffle/blob/launch/frontend/src/assets/img/icpl_logo.png)
@@ -59,8 +87,13 @@ https://shuffler.io
## License
All modular information related to Shuffle will be under MIT (anyone can use it for whatever purpose), with Shuffle itself using AGPLv3.
Apps & App SDK: MIT
Workflows: MIT
Documentation: MIT
Shuffle backend: AGPLv3
Apps, specification and App SDK: MIT
## Architecture
![Shuffle Architecture](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_architecture.png)
### Repository overview
Below is the folder structure with a short explanation
@@ -68,17 +101,14 @@ Below is the folder structure with a short explanation
├── README.md # What you're reading right now
├── backend # Contains backend related code.
│   ├── go-app # The backend golang webserver
│   ├── app_gen # Code for app generation outside the Shuffle platform
│ └── app_sdk # The SDK used for apps
├── frontend # Contains frontend code. ReactJS and cytoscape. Horrible code :)
├── functions # Contains google cloud function code mainly.
│   ├── static_baseline.py # Static code used by stitcher.go to generate code
│   ├── stitcher.go # Attempts to stitch together an app - part of backend now
├── frontend # Contains frontend code. ReactJS, Material UI and cytoscape
├── functions # Has execution and extension resources, such as the Wazuh integration
│   ├── onprem # Code for onprem solutions
│  │   ├── Orborus # Distributes execution locations
│  │   ├── Worker # Runs a workflow
└ docker-compose.yml # Used for deployments
```
**It's in BETA** - [Get in touch](https://shuffler.io/contact), send a mail to [frikky@shuffler.io](mailto:frikky@shuffler.io) or poke me on twitter [@frikkylikeme](https://twitter.com/frikkylikeme)
**It's in BETA (0.8.60)** - [Get in touch](https://shuffler.io/contact), send a mail to [frikky@shuffler.io](mailto:frikky@shuffler.io) or poke me on twitter [@frikkylikeme](https://twitter.com/frikkylikeme)
SECURITY.md
+19
View File
@@ -0,0 +1,19 @@
# Security Policy
## Supported Versions
Shuffle is currently still in beta, but we aim to support older version with critical severity issues, but do advise you to stay up to date with Major versions.
| Version | Supported |
| ------- | ------------------ |
| >=0.9.0 | :white_check_mark: |
| < 0.9.0 | :x: |
## Reporting a Vulnerability
Reporting a vulnerability can either be done to (frikky@shuffler.io)[mailto:frikky@shuffler.io] or [through the contact page on our website](https://shuffler.io/contact)
Security.txt: https://shuffler.io/.well_known/security.txt
When a >medium severity vulnerability is discovered, expect it to be fixed ASAP - please nag us until it is. Security is a top priority, and we expect you to keep us accountable.
In the case it makes sense, we'll further create a security advisory, and publish a new CVE for your new glorious finding.
backend/Dockerfile
+2 -11
View File
@@ -1,4 +1,4 @@
from golang as builder
FROM golang:1.17.2-buster as builder
# Add files
RUN mkdir /app
@@ -7,18 +7,11 @@ WORKDIR /app
ADD ./go-app/main.go /app
ADD ./go-app/walkoff.go /app
ADD ./go-app/docker.go /app
ADD ./go-app/codegen.go /app
ADD ./go-app/files.go /app
ADD ./go-app/go.mod /app
# Required files for code generation
ADD ./app_sdk/app_base.py /app_sdk
ADD ./app_sdk/static_baseline.py /app_sdk
ADD ./app_sdk_kali/app_base.py /app_sdk_kali
ADD ./app_sdk_kali/static_baseline.py /app_sdk_kali
ADD ./app_sdk_blackarch/app_base.py /app_sdk_blackarch
ADD ./app_sdk_blackarch/static_baseline.py /app_sdk_blackarch
ADD ./app_gen /app_gen
RUN go get -v
@@ -29,12 +22,10 @@ RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o webapp .
FROM alpine:latest as certs
RUN apk --update add ca-certificates
FROM alpine:3.12
FROM alpine:3.14.2
COPY --from=builder /app/ /app
COPY --from=builder /app_sdk/ /app_sdk
COPY --from=builder /app_sdk_kali/ /app_sdk_kali
COPY --from=builder /app_sdk_blackarch/ /app_sdk_blackarch
COPY --from=builder /app_gen/ /app_gen
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
backend/app_gen/openapi-parsers/other/TIO-API-Container-Security-v1.json
-1
View File
File diff suppressed because one or more lines are too long
backend/app_gen/openapi-parsers/other/TIO-API-Container-Security-v2.json
-1
View File
File diff suppressed because one or more lines are too long
backend/app_gen/openapi-parsers/other/TIO-API-Downloads-API.json
-1
View File
File diff suppressed because one or more lines are too long
backend/app_gen/openapi-parsers/other/TIO-API-Tenable-Platform.json
-1
View File
File diff suppressed because one or more lines are too long
backend/app_gen/openapi-parsers/other/TIO-API-Vulnerability-Management.json
-1
View File
File diff suppressed because one or more lines are too long
backend/app_gen/openapi-parsers/other/TIO-API-Web-Application-Scanning.json
-1
View File
File diff suppressed because one or more lines are too long
backend/app_gen/openapi/baseline/Dockerfile
+14 -11
View File
@@ -2,24 +2,27 @@
FROM frikky/shuffle:app_sdk as base
# We're going to stage away all of the bloat from the build tools so lets create a builder stage
FROM base as builder
#FROM base as builder
# Install all alpine build tools needed for our pip installs
RUN apk --no-cache add --update alpine-sdk libffi libffi-dev musl-dev openssl-dev
#RUN apk --no-cache add --update alpine-sdk libffi libffi-dev musl-dev openssl-dev
# Install all of our pip packages in a single directory that we can copy to our base image later
RUN mkdir /install
WORKDIR /install
COPY requirements.txt /requirements.txt
RUN pip install --prefix="/install" -r /requirements.txt
#RUN mkdir /install
#WORKDIR /install
#COPY requirements.txt /requirements.txt
#
## Switch back to our base image and copy in all of our built packages and source code
#FROM base
#COPY --from=builder /install /usr/local
# Switch back to our base image and copy in all of our built packages and source code
FROM base
COPY --from=builder /install /usr/local
COPY src /app
# Install any binary dependencies needed in our final image - this can be a lot of different stuff
RUN apk --no-cache add --update libmagic
#RUN apk --no-cache add --update libmagic
WORKDIR /
COPY requirements.txt /requirements.txt
RUN pip install --prefix="/usr/local" -r /requirements.txt
COPY src /app
# Finally, lets run our app!
WORKDIR /app
backend/app_sdk/Dockerfile
+9 -5
View File
@@ -1,16 +1,20 @@
FROM python:3.7-alpine as base
#FROM python:3.9.1-alpine as base
FROM python:3.10.0-alpine as base
FROM base as builder
RUN apk --no-cache add --update alpine-sdk libffi libffi-dev musl-dev openssl-dev
RUN apk --no-cache add --update alpine-sdk libffi libffi-dev musl-dev openssl-dev tzdata coreutils
RUN mkdir /install
WORKDIR /install
FROM base
#--no-cache
RUN apk update && apk add --update tzdata libmagic alpine-sdk libffi libffi-dev musl-dev openssl-dev coreutils
COPY --from=builder /install /usr/local
COPY requirements.txt /requirements.txt
RUN pip3 install -r /requirements.txt
FROM base
COPY --from=builder /install /usr/local
COPY __init__.py /app/walkoff_app_sdk/__init__.py
COPY app_base.py /app/walkoff_app_sdk/app_base.py
backend/app_sdk/Dockerfile_alpine_grpc
+42
View File
@@ -0,0 +1,42 @@
FROM python:3.10.0-alpine as base
FROM base as builder
RUN apk --no-cache add --update \
alpine-sdk \
build-base \
g++ \
gcc \
libffi \
libffi-dev \
libstdc++ \
linux-headers \
musl-dev \
openssl-dev \
tzdata \
coreutils
RUN pip install --upgrade pip && \
pip install --prefix="/install" --no-cache-dir grpcio grpcio-tools && \
apk del --purge \
g++ \
gcc \
musl-dev \
libffi-dev \
libstdc++ \
build-base \
linux-headers
RUN mkdir -p /install
WORKDIR /install
FROM base
#--no-cache
RUN apk update && apk add --update tzdata libmagic alpine-sdk libffi libffi-dev musl-dev openssl-dev coreutils
COPY --from=builder /install /usr/local
COPY requirements.txt /requirements.txt
RUN pip3 install -r /requirements.txt
COPY __init__.py /app/walkoff_app_sdk/__init__.py
COPY app_base.py /app/walkoff_app_sdk/app_base.py
backend/app_sdk_blackarch/Dockerfile → backend/app_sdk/Dockerfile_blackarch
View File
backend/app_sdk_kali/Dockerfile → backend/app_sdk/Dockerfile_kali
View File
backend/app_sdk/Dockerfile_ubuntu
+22
View File
@@ -0,0 +1,22 @@
FROM ubuntu as base
FROM base as builder
RUN apt-get update
RUN apt-get dist-upgrade -y
RUN apt install build-essential libffi-dev musl-dev openssl python3 python3-pip -y
RUN mkdir /install
WORKDIR /install
COPY requirements.txt /requirements.txt
RUN pip install --prefix="/install" -r /requirements.txt
FROM base
RUN apt-get update
RUN apt-get dist-upgrade -y
RUN apt install build-essential libffi-dev musl-dev openssl python3 python3-pip -y
COPY --from=builder /install /usr/local
COPY __init__.py /app/walkoff_app_sdk/__init__.py
COPY app_base.py /app/walkoff_app_sdk/app_base.py
backend/app_sdk/README.md
+10 -4
View File
@@ -1,9 +1,5 @@
# app_sdk.py
This is the SDK used for apps to behave like they should.
To change it in the backend, upload it to Buckets/shuffler.appspot.com/generated_apps/baseline.
# static_baseline.py
It's used for python code generation and should be under MIT. Has to be located here because it's used by the backend.
## If you want to update apps.. PS: downloads from docker hub do overrides.. :)
1. Write your code & check if runtime works
@@ -12,5 +8,15 @@ It's used for python code generation and should be under MIT. Has to be located
4. Delete the specific app's Docker image (docker rmi frikky/shuffle:...)
5. Rebuild the Docker image (click load in GUI?)
## Cloud updates
1. Go to shuffle cloud on GCP
2. Go to Cloud Storage
3. Find shuffler.appspot.com
4. Navigate to generated_apps/baseline
5. Update SDK there. This will make all new apps run with the new SDK
## Cloud app force-updates
1. Run the "stitcher.go" program in the public shuffle-shared repository.
# LICENSE
Everything in here is MIT, not AGPLv3 as indicated by the license.
backend/app_sdk/app_base.py
+2208 -580
View File
File diff suppressed because it is too large Load Diff
backend/app_sdk/build.sh
+39 -7
View File
@@ -1,14 +1,46 @@
#!/bin/bash
### DEFAULT
NAME=shuffle-app_sdk
VERSION=0.8.54
VERSION=0.9.70
docker rmi docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION --force
docker build . -t frikky/shuffle:app_sdk -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
#docker push frikky/$NAME:$VERSION
#docker push docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION
#docker push ghcr.io/frikky/$NAME:$VERSION
#docker tag ghcr.io/frikky/$NAME:$VERSION frikky/shuffle:app_sdk
docker build . -f Dockerfile -t frikky/shuffle:app_sdk -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION -t ghcr.io/frikky/$NAME:nightly
docker push frikky/shuffle:app_sdk
docker push ghcr.io/frikky/$NAME:$VERSION
docker push ghcr.io/frikky/$NAME:nightly
docker push ghcr.io/frikky/$NAME:latest
#### UBUNTU
NAME=shuffle-app_sdk_ubuntu
docker build . -f Dockerfile_ubuntu -t frikky/shuffle:app_sdk_ubuntu -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
docker push frikky/shuffle:app_sdk_ubuntu
docker push ghcr.io/frikky/$NAME:$VERSION
#### Alpine GRPC
NAME=shuffle-app_sdk_grpc
docker build . -f Dockerfile_alpine_grpc -t frikky/shuffle:app_sdk_grpc -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
docker push frikky/shuffle:app_sdk_grpc
docker push ghcr.io/frikky/$NAME:$VERSION
#### KALI ###
#NAME=shuffle-app_sdk_kali
#docker build . -f Dockerfile_kali -t frikky/shuffle:app_sdk_kali -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
#
#docker push frikky/shuffle:app_sdk_kali
#docker push ghcr.io/frikky/$NAME:$VERSION
#docker push ghcr.io/frikky/$NAME:nightly
### BLACKARCH ###
#NAME=shuffle-app_sdk_blackarch
#docker build . -f Dockerfile_blackarch -t frikky/shuffle:app_sdk_blackarch -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
#
#docker push frikky/shuffle:app_sdk_blackarch
#docker push ghcr.io/frikky/$NAME:$VERSION
#docker push ghcr.io/frikky/$NAME:nightly
backend/app_sdk/requirements.txt
+7 -2
View File
@@ -1,2 +1,7 @@
urllib3
requests
urllib3==1.26.5
requests==2.25.1
MarkupSafe==2.0.1
liquidpy==0.7.3
flask[async]==2.0.2
waitress==2.1.0
#flask==1.1.2
backend/app_sdk/static_baseline.py
-76
View File
@@ -1,76 +0,0 @@
import os
import sys
import time
import logging
import requests
# Goal here:
# * Make an app from WALKOFF able to run without app_base.py from WALKOFF
# # How:
# * Make it rely 100% on INPUT throug HTTP invocations instead of redis READS
# # But really, how?
# * Make a WORKER that reads the queue, and reuses a function
# Here to get it global
apikey = ""
try:
apikey = os.environ["FUNCTION_APIKEY"]
except KeyError:
pass
# Authorize the execution
def authorization(request):
# This is basically my issue, but it enforces the use of an internal API key for execution
try:
apikey = os.environ["FUNCTION_APIKEY"]
except KeyError:
return f"Internal server error", 500
# Check API key from ENV authentication
authentication = request.headers.get("Authorization")
if authentication == None: return f"Unauthorized", 401
apikey_split = authentication.split(" ")
if apikey_split[0] != "Bearer" or len(apikey_split) != 2:
return f"Apikey error", 401
if apikey != apikey_split[1]:
return f"Unauthorized", 401
return run(request)
class AppBase:
""" The base class for Python-based Walkoff applications, handles Redis and logging configurations. """
__version__ = None
app_name = None
def __init__(self, redis=None, logger=None, console_logger=None):#, docker_client=None):
self.logger = logger if logger is not None else logging.getLogger("AppBaseLogger")
self.redis=redis
self.console_logger=console_logger
self.current_execution_id = None
self.url = "https://shuffler.io"
self.apikey = apikey
@classmethod
async def run(cls, action):
""" Connect to Redis and HTTP session, await actions """
logging.basicConfig(format="{asctime} - {name} - {levelname}:{message}", style='{')
logger = logging.getLogger(f"{cls.__name__}")
logger.setLevel(logging.DEBUG)
app = cls(redis=None, logger=logger, console_logger=logger)
# Authorization for the app/function to control the workflow
# Function will crash if its wrong, which it probably should.
await app.execute_action(action)
async def execute_action(self, action):
# FIXME - add request for the function STARTING here. Use "results stream" or something
# PAUSED, AWAITING_DATA, PENDING, COMPLETED, ABORTED, EXECUTING, SUCCESS, FAILURE
self.authorization = action["authorization"]
self.execution_id = action["execution_id"]
self.current_execution_id = action["execution_id"]
backend/app_sdk_blackarch/LICENSE
-21
View File
@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2020 Frikkylikeme
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
backend/app_sdk_blackarch/README.md
-16
View File
@@ -1,16 +0,0 @@
# app_sdk.py
This is the SDK used for apps to behave like they should.
To change it in the backend, upload it to Buckets/shuffler.appspot.com/generated_apps/baseline.
# static_baseline.py
It's used for python code generation and should be under MIT. Has to be located here because it's used by the backend.
## If you want to update apps.. PS: downloads from docker hub do overrides.. :)
1. Write your code & check if runtime works
2. Build app_base image
3. docker rm $(docker ps -aq) # Remove all stopped containers
4. Delete the specific app's Docker image (docker rmi frikky/shuffle:...)
5. Rebuild the Docker image (click load in GUI?)
# LICENSE
Everything in here is MIT, not AGPLv3 as indicated by the license.
backend/app_sdk_blackarch/app_base.py
-1169
View File
File diff suppressed because it is too large Load Diff
backend/app_sdk_blackarch/build.sh
-13
View File
@@ -1,13 +0,0 @@
#!/bin/bash
NAME=app_sdk_blackarch
VERSION=0.7.3
docker rmi docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION --force
docker build . -t frikky/shuffle:$NAME -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
#docker push frikky/$NAME:$VERSION
#docker push docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION
#docker push ghcr.io/frikky/$NAME:$VERSION
docker push frikky/shuffle:$NAME
docker push ghcr.io/frikky/$NAME:$VERSION
backend/app_sdk_blackarch/static_baseline.py
-76
View File
@@ -1,76 +0,0 @@
import os
import sys
import time
import logging
import requests
# Goal here:
# * Make an app from WALKOFF able to run without app_base.py from WALKOFF
# # How:
# * Make it rely 100% on INPUT throug HTTP invocations instead of redis READS
# # But really, how?
# * Make a WORKER that reads the queue, and reuses a function
# Here to get it global
apikey = ""
try:
apikey = os.environ["FUNCTION_APIKEY"]
except KeyError:
pass
# Authorize the execution
def authorization(request):
# This is basically my issue, but it enforces the use of an internal API key for execution
try:
apikey = os.environ["FUNCTION_APIKEY"]
except KeyError:
return f"Internal server error", 500
# Check API key from ENV authentication
authentication = request.headers.get("Authorization")
if authentication == None: return f"Unauthorized", 401
apikey_split = authentication.split(" ")
if apikey_split[0] != "Bearer" or len(apikey_split) != 2:
return f"Apikey error", 401
if apikey != apikey_split[1]:
return f"Unauthorized", 401
return run(request)
class AppBase:
""" The base class for Python-based Walkoff applications, handles Redis and logging configurations. """
__version__ = None
app_name = None
def __init__(self, redis=None, logger=None, console_logger=None):#, docker_client=None):
self.logger = logger if logger is not None else logging.getLogger("AppBaseLogger")
self.redis=redis
self.console_logger=console_logger
self.current_execution_id = None
self.url = "https://shuffler.io"
self.apikey = apikey
@classmethod
async def run(cls, action):
""" Connect to Redis and HTTP session, await actions """
logging.basicConfig(format="{asctime} - {name} - {levelname}:{message}", style='{')
logger = logging.getLogger(f"{cls.__name__}")
logger.setLevel(logging.DEBUG)
app = cls(redis=None, logger=logger, console_logger=logger)
# Authorization for the app/function to control the workflow
# Function will crash if its wrong, which it probably should.
await app.execute_action(action)
async def execute_action(self, action):
# FIXME - add request for the function STARTING here. Use "results stream" or something
# PAUSED, AWAITING_DATA, PENDING, COMPLETED, ABORTED, EXECUTING, SUCCESS, FAILURE
self.authorization = action["authorization"]
self.execution_id = action["execution_id"]
self.current_execution_id = action["execution_id"]
backend/app_sdk_kali/LICENSE
-21
View File
@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2020 Frikkylikeme
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
backend/app_sdk_kali/README.md
-16
View File
@@ -1,16 +0,0 @@
# app_sdk.py
This is the SDK used for apps to behave like they should.
To change it in the backend, upload it to Buckets/shuffler.appspot.com/generated_apps/baseline.
# static_baseline.py
It's used for python code generation and should be under MIT. Has to be located here because it's used by the backend.
## If you want to update apps.. PS: downloads from docker hub do overrides.. :)
1. Write your code & check if runtime works
2. Build app_base image
3. docker rm $(docker ps -aq) # Remove all stopped containers
4. Delete the specific app's Docker image (docker rmi frikky/shuffle:...)
5. Rebuild the Docker image (click load in GUI?)
# LICENSE
Everything in here is MIT, not AGPLv3 as indicated by the license.
backend/app_sdk_kali/app_base.py
-1169
View File
File diff suppressed because it is too large Load Diff
backend/app_sdk_kali/build.sh
-13
View File
@@ -1,13 +0,0 @@
#!/bin/bash
NAME=app_sdk_kali
VERSION=0.7.3
docker rmi docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION --force
docker build . -t frikky/shuffle:$NAME -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
#docker push frikky/$NAME:$VERSION
#docker push docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION
#docker push ghcr.io/frikky/$NAME:$VERSION
docker push frikky/shuffle:$NAME
docker push ghcr.io/frikky/$NAME:$VERSION
backend/app_sdk_kali/requirements.txt
-2
View File
@@ -1,2 +0,0 @@
requests
urllib3
backend/app_sdk_kali/static_baseline.py
-76
View File
@@ -1,76 +0,0 @@
import os
import sys
import time
import logging
import requests
# Goal here:
# * Make an app from WALKOFF able to run without app_base.py from WALKOFF
# # How:
# * Make it rely 100% on INPUT throug HTTP invocations instead of redis READS
# # But really, how?
# * Make a WORKER that reads the queue, and reuses a function
# Here to get it global
apikey = ""
try:
apikey = os.environ["FUNCTION_APIKEY"]
except KeyError:
pass
# Authorize the execution
def authorization(request):
# This is basically my issue, but it enforces the use of an internal API key for execution
try:
apikey = os.environ["FUNCTION_APIKEY"]
except KeyError:
return f"Internal server error", 500
# Check API key from ENV authentication
authentication = request.headers.get("Authorization")
if authentication == None: return f"Unauthorized", 401
apikey_split = authentication.split(" ")
if apikey_split[0] != "Bearer" or len(apikey_split) != 2:
return f"Apikey error", 401
if apikey != apikey_split[1]:
return f"Unauthorized", 401
return run(request)
class AppBase:
""" The base class for Python-based Walkoff applications, handles Redis and logging configurations. """
__version__ = None
app_name = None
def __init__(self, redis=None, logger=None, console_logger=None):#, docker_client=None):
self.logger = logger if logger is not None else logging.getLogger("AppBaseLogger")
self.redis=redis
self.console_logger=console_logger
self.current_execution_id = None
self.url = "https://shuffler.io"
self.apikey = apikey
@classmethod
async def run(cls, action):
""" Connect to Redis and HTTP session, await actions """
logging.basicConfig(format="{asctime} - {name} - {levelname}:{message}", style='{')
logger = logging.getLogger(f"{cls.__name__}")
logger.setLevel(logging.DEBUG)
app = cls(redis=None, logger=logger, console_logger=logger)
# Authorization for the app/function to control the workflow
# Function will crash if its wrong, which it probably should.
await app.execute_action(action)
async def execute_action(self, action):
# FIXME - add request for the function STARTING here. Use "results stream" or something
# PAUSED, AWAITING_DATA, PENDING, COMPLETED, ABORTED, EXECUTING, SUCCESS, FAILURE
self.authorization = action["authorization"]
self.execution_id = action["execution_id"]
self.current_execution_id = action["execution_id"]
backend/database/Dockerfile
-5
View File
@@ -1,5 +0,0 @@
# 2. docker run -p 8000:8000
FROM google/cloud-sdk
EXPOSE 8000
CMD ["gcloud", "beta", "emulators", "datastore", "start", "--project=shuffle", "--host-port", "0.0.0.0:8000", "--data-dir=/etc/shuffle"]
backend/database/opensearch/docker-compose.yml
+34
View File
@@ -0,0 +1,34 @@
version: '3'
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
hostname: shuffle-database
container_name: shuffle-opensearch
environment:
- cluster.name=shuffle-cluster
- node.name=shuffle-opensearch
- discovery.seed_hosts=shuffle-opensearch
- cluster.initial_master_nodes=shuffle-opensearch
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
- cluster.routing.allocation.disk.threshold_enabled=false
- opendistro_security.disabled=true
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- ~/git/shuffle/shuffle-database:/usr/share/opensearch/data
ports:
- 9200:9200
networks:
- opensearch-net
volumes:
opensearch-data1:
networks:
opensearch-net:
backend/go-app/app.yaml
-98
View File
@@ -1,98 +0,0 @@
runtime: go111
env_variables:
automatic_scaling:
max_instances: 1
min_instances: 1
handlers:
- url: /api/(.*)
script: auto
secure: always
- url: /static/js/(.*)
static_files: build/static/js/\1
upload: build/static/js/(.*)
secure: always
- url: /static/css/(.*)
static_files: build/static/css/\1
upload: build/static/css/(.*)
secure: always
- url: /images/(.*)
static_files: build/images/\1
upload: build/images/(.*)
secure: always
- url: /(.*\.(json|ico))$
static_files: build/\1
upload: build/.*\.(json|ico)$
secure: always
- url: /manifest.json
static_files: build/manifest.json
upload: build/manifest.json
secure: always
# lol.. wildcard doesn't work with /api/(.*) for some reason
- url: /
static_files: build/index.html
upload: build/index.html
secure: always
- url: /home
static_files: build/index.html
upload: build/index.html
secure: always
- url: /passwordreset
static_files: build/index.html
upload: build/index.html
secure: always
- url: /login
static_files: build/index.html
upload: build/index.html
secure: always
- url: /register
static_files: build/index.html
upload: build/index.html
secure: always
- url: /workflows
static_files: build/index.html
upload: build/index.html
secure: always
- url: /workflows/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
- url: /info/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
- url: /docs/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
- url: /docs
static_files: build/index.html
upload: build/index.html
secure: always
- url: /settings
static_files: build/index.html
upload: build/index.html
secure: always
- url: /apps
static_files: build/index.html
upload: build/index.html
secure: always
- url: /contact
static_files: build/index.html
upload: build/index.html
secure: always
- url: /apps/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
- url: /register/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
- url: /passwordreset/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
backend/go-app/codegen.go
-1959
View File
File diff suppressed because it is too large Load Diff
backend/go-app/docker.go
+303 -287
View File
@@ -2,21 +2,28 @@ package main
// Docker
import (
"github.com/shuffle/shuffle-shared"
"archive/tar"
//"bufio"
"path/filepath"
//"strconv"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
//"github.com/docker/docker"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
//"github.com/docker/docker/api/types/container"
"github.com/docker/docker/client"
newdockerclient "github.com/fsouza/go-dockerclient"
"github.com/go-git/go-billy/v5"
network "github.com/docker/docker/api/types/network"
natting "github.com/docker/go-connections/nat"
//network "github.com/docker/docker/api/types/network"
//natting "github.com/docker/go-connections/nat"
"io"
"io/ioutil"
@@ -24,7 +31,6 @@ import (
"net/http"
"os"
"strings"
//"google.golang.org/appengine"
)
// Parses a directory with a Dockerfile into a tar for Docker images..
@@ -213,11 +219,27 @@ func buildImageMemory(fs billy.Filesystem, tags []string, dockerfileFolder strin
// Dockerfile is inside the TAR itself. Not local context
// docker build --build-arg http_proxy=http://my.proxy.url
// Attempt at setting name according to #359: https://github.com/frikky/Shuffle/issues/359
labels := map[string]string{}
//target := ""
//if len(tags) > 0 {
// if strings.Contains(tags[0], ":") {
// version := strings.Split(tags[0], ":")
// if len(version) == 2 {
// target = fmt.Sprintf("shuffle-build-%s", version[1])
// tags = append(tags, target)
// labels["name"] = target
// }
// }
//}
buildOptions := types.ImageBuildOptions{
Remove: true,
Tags: tags,
BuildArgs: map[string]*string{},
Labels: labels,
}
// NetworkMode: "host",
httpProxy := os.Getenv("HTTP_PROXY")
@@ -230,55 +252,59 @@ func buildImageMemory(fs billy.Filesystem, tags []string, dockerfileFolder strin
}
// Build the actual image
log.Printf("[INFO] Building %s. This may take up to a few minutes.", dockerfileFolder)
log.Printf(`[INFO] Building %s with proxy "%s". Tags: "%s". This may take up to a few minutes.`, dockerfileFolder, httpsProxy, strings.Join(tags, ","))
imageBuildResponse, err := client.ImageBuild(
ctx,
dockerFileTarReader,
buildOptions,
)
//log.Printf("RESPONSE: %#v", imageBuildResponse)
//log.Printf("Response: %#v", imageBuildResponse.Body)
//log.Printf("IMAGERESPONSE: %#v", imageBuildResponse.Body)
//log.Printf("[DEBUG] IMAGERESPONSE: %#v", imageBuildResponse.Body)
defer imageBuildResponse.Body.Close()
buildBuf := new(strings.Builder)
_, newerr := io.Copy(buildBuf, imageBuildResponse.Body)
if newerr != nil {
log.Printf("Failed reading Docker build STDOUT: %s", newerr)
} else {
log.Printf("STRING: %s", buildBuf.String())
if strings.Contains(buildBuf.String(), "errorDetail") {
log.Printf("[ERROR] Docker build:\n%s\nERROR ABOVE: Trying to pull tags from: %s", buildBuf.String(), strings.Join(tags, "\n"))
if imageBuildResponse.Body != nil {
defer imageBuildResponse.Body.Close()
buildBuf := new(strings.Builder)
_, newerr := io.Copy(buildBuf, imageBuildResponse.Body)
if newerr != nil {
log.Printf("[WARNING] Failed reading Docker build STDOUT: %s", newerr)
} else {
log.Printf("[INFO] STRING: %s", buildBuf.String())
if strings.Contains(buildBuf.String(), "errorDetail") {
log.Printf("[ERROR] Docker build:\n%s\nERROR ABOVE: Trying to pull tags from: %s", buildBuf.String(), strings.Join(tags, "\n"))
// Handles pulling of the same image if applicable
// This fixes some issues with older versions of Docker which can't build
// on their own ( <17.05 )
pullOptions := types.ImagePullOptions{}
downloaded := false
for _, image := range tags {
// Is this ok? Not sure. Tags shouldn't be controlled here prolly.
image = strings.ToLower(image)
// Handles pulling of the same image if applicable
// This fixes some issues with older versions of Docker which can't build
// on their own ( <17.05 )
pullOptions := types.ImagePullOptions{}
downloaded := false
for _, image := range tags {
// Is this ok? Not sure. Tags shouldn't be controlled here prolly.
image = strings.ToLower(image)
newImage := fmt.Sprintf("%s/%s", registryName, image)
log.Printf("[INFO] Pulling image %s", newImage)
reader, err := client.ImagePull(ctx, newImage, pullOptions)
if err != nil {
log.Printf("[ERROR] Failed getting image %s: %s", newImage, err)
continue
newImage := fmt.Sprintf("%s/%s", registryName, image)
log.Printf("[INFO] Pulling image %s", newImage)
reader, err := client.ImagePull(ctx, newImage, pullOptions)
if err != nil {
log.Printf("[ERROR] Failed getting image %s: %s", newImage, err)
continue
}
// Attempt to retag the image to not contain registry...
//newBuf := buildBuf
downloaded = true
io.Copy(os.Stdout, reader)
log.Printf("[INFO] Successfully downloaded and built %s", newImage)
}
// Attempt to retag the image to not contain registry...
if !downloaded {
//newBuf := buildBuf
downloaded = true
io.Copy(os.Stdout, reader)
log.Printf("[INFO] Successfully downloaded and built %s", newImage)
return errors.New(fmt.Sprintf("Failed to build / download images %s", strings.Join(tags, ",")))
}
//baseDockerName
}
if !downloaded {
return errors.New(fmt.Sprintf("Failed to build / download images %s", strings.Join(tags, ",")))
}
//baseDockerName
}
}
@@ -299,7 +325,7 @@ func buildImage(tags []string, dockerfileFolder string) error {
return err
}
log.Printf("Tags: %s", tags)
log.Printf("[INFO] Docker Tags: %s", tags)
dockerfileSplit := strings.Split(dockerfileFolder, "/")
// Create a buffer
@@ -390,92 +416,9 @@ func stopWebhook(image string, identifier string) error {
return nil
}
// FIXME - remember to set DOCKER_API_VERSION
// FIXME - remove github.com/docker/docker/vendor
// FIXME - Library dependencies for NAT is fucked..
// https://docs.docker.com/develop/sdk/examples/
func deployWebhook(image string, identifier string, path string, port string, callbackurl string, apikey string) error {
cli, err := client.NewEnvClient()
if err != nil {
fmt.Println("Unable to create docker client")
return err
}
newport, err := natting.NewPort("tcp", port)
if err != nil {
fmt.Println("Unable to create docker port")
return err
}
// FIXME - logging?
hostConfig := &container.HostConfig{
PortBindings: natting.PortMap{
newport: []natting.PortBinding{
{
HostIP: "0.0.0.0",
HostPort: port,
},
},
},
RestartPolicy: container.RestartPolicy{
Name: "always",
},
LogConfig: container.LogConfig{
Type: "json-file",
Config: map[string]string{},
},
}
//networkConfig := &network.NetworkSettings{}
networkConfig := &network.NetworkingConfig{
EndpointsConfig: map[string]*network.EndpointSettings{},
}
test := &network.EndpointSettings{
Gateway: "helo",
}
networkConfig.EndpointsConfig["bridge"] = test
exposedPorts := map[natting.Port]struct{}{
newport: struct{}{},
}
config := &container.Config{
Image: image,
Env: []string{
fmt.Sprintf("URIPATH=%s", path),
fmt.Sprintf("HOOKPORT=%s", port),
fmt.Sprintf("CALLBACKURL=%s", callbackurl),
fmt.Sprintf("APIKEY=%s", apikey),
fmt.Sprintf("HOOKID=%s", identifier),
},
ExposedPorts: exposedPorts,
Hostname: fmt.Sprintf("%s-%s", image, identifier),
}
cont, err := cli.ContainerCreate(
context.Background(),
config,
hostConfig,
networkConfig,
fmt.Sprintf("%s-%s", image, identifier),
)
if err != nil {
log.Println(err)
return err
}
cli.ContainerStart(context.Background(), cont.ID, types.ContainerStartOptions{})
log.Printf("Container %s is created", cont.ID)
return nil
}
// Starts a new webhook
func handleStopHookDocker(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
cors := shuffle.HandleCors(resp, request)
if cors {
return
}
@@ -500,7 +443,7 @@ func handleStopHookDocker(resp http.ResponseWriter, request *http.Request) {
}
ctx := context.Background()
hook, err := getHook(ctx, fileId)
hook, err := shuffle.GetHook(ctx, fileId)
if err != nil {
log.Printf("Failed getting hook %s (stop docker): %s", fileId, err)
resp.WriteHeader(401)
@@ -520,8 +463,8 @@ func handleStopHookDocker(resp http.ResponseWriter, request *http.Request) {
hook.Status = "stopped"
hook.Running = false
hook.Actions = []HookAction{}
err = setHook(ctx, *hook)
hook.Actions = []shuffle.HookAction{}
err = shuffle.SetHook(ctx, *hook)
if err != nil {
log.Printf("Failed setting hook: %s", err)
resp.WriteHeader(401)
@@ -560,7 +503,7 @@ var webhook = `{
// Starts a new webhook
func handleDeleteHookDocker(resp http.ResponseWriter, request *http.Request) {
ctx := context.Background()
cors := handleCors(resp, request)
cors := shuffle.HandleCors(resp, request)
if cors {
return
}
@@ -584,7 +527,7 @@ func handleDeleteHookDocker(resp http.ResponseWriter, request *http.Request) {
return
}
err := DeleteKey(ctx, "hooks", fileId)
err := shuffle.DeleteKey(ctx, "hooks", fileId)
if err != nil {
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "message": "Can't delete"}`))
@@ -605,121 +548,6 @@ func handleDeleteHookDocker(resp http.ResponseWriter, request *http.Request) {
resp.Write([]byte(`{"success": true, "message": "Deleted webhook"}`))
}
// Starts a new webhook
func handleStartHookDocker(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
location := strings.Split(request.URL.String(), "/")
var fileId string
if location[1] == "api" {
if len(location) <= 4 {
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
fileId = location[4]
}
if len(fileId) != 32 {
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "message": "ID not valid"}`))
return
}
ctx := context.Background()
hook, err := getHook(ctx, fileId)
if err != nil {
log.Printf("Failed getting hook %s (start docker): %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
if len(hook.Info.Url) == 0 {
log.Printf("Hook url can't be empty.")
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
log.Printf("Status: %s", hook.Status)
log.Printf("Running: %t", hook.Running)
if hook.Running || hook.Status == "Running" {
message := fmt.Sprintf("Error: %s is already running", hook.Id)
log.Println(message)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "%s"}`, message)))
return
}
// FIXME - verify?
// FIXME - static port? Generate from available range.
image := "webhook"
filepath := "/webhook"
baseUrl := "http://localhost"
callbackUrl := "http://localhost:8001"
// This is here to force stop and remove the old webhook
err = stopWebhook(image, fileId)
if err != nil {
log.Printf("Container stop issue for %s-%s: %s", image, fileId, err)
}
// Dynamic ish ports
var startPort int64 = 5001
var endPort int64 = 5010
port := findAvailablePorts(startPort, endPort)
if len(port) == 0 {
message := fmt.Sprintf("Not ports available in the range %d-%d", startPort, endPort)
log.Println(message)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "%s"}`, message)))
return
}
hook.Status = "running"
hook.Running = true
// Set this for more than just hooks?
if hook.Type == "webhook" {
hook.Info.Url = fmt.Sprintf("%s:%s%s", baseUrl, port, filepath)
}
err = setHook(ctx, *hook)
if err != nil {
log.Printf("Failed setting hook: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
// Cloud run? Let's make a generic webhook that can be deployed easily
log.Printf("Should run a webhook with the following: \nUrl: %s\nId: %s\n", hook.Info.Url, hook.Id)
// FIXME - set port based on what the user specified / what was generated
// FIXME - add nonstatic APIKEY
apiKey := "ASD"
err = deployWebhook(image, fileId, filepath, port, callbackUrl, apiKey)
if err != nil {
log.Printf("Failed starting container %s-%s: %s", image, fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
// FIXME - get some real data?
log.Printf("[INFO] Successfully started %s-%s on port %s with filepath %s", image, fileId, port, filepath)
resp.WriteHeader(200)
resp.Write([]byte(`{"success": true, "message": "Started webhook"}`))
return
}
// Checks if an image exists
func imageCheckBuilder(images []string) error {
//log.Printf("[FIXME] ImageNames to check: %#v", images)
@@ -766,7 +594,7 @@ func imageCheckBuilder(images []string) error {
}
func hookTest() {
var hook Hook
var hook shuffle.Hook
err := json.Unmarshal([]byte(webhook), &hook)
log.Println(webhook)
if err != nil {
@@ -775,12 +603,12 @@ func hookTest() {
}
ctx := context.Background()
err = setHook(ctx, hook)
err = shuffle.SetHook(ctx, hook)
if err != nil {
log.Printf("Failed setting hook: %s", err)
}
returnHook, err := getHook(ctx, hook.Id)
returnHook, err := shuffle.GetHook(ctx, hook.Id)
if err != nil {
log.Printf("Failed getting hook %s (test): %s", hook.Id, err)
}
@@ -792,19 +620,19 @@ func hookTest() {
//https://stackoverflow.com/questions/23935141/how-to-copy-docker-images-from-one-host-to-another-without-using-a-repository
func getDockerImage(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
cors := shuffle.HandleCors(resp, request)
if cors {
return
}
// Just here to verify that the user is logged in
_, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("Api authentication failed in validate swagger: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
//_, err := shuffle.HandleApiAuthentication(resp, request)
//if err != nil {
// log.Printf("[WARNING] Api authentication failed in DOWNLOAD IMAGE: %s", err)
// resp.WriteHeader(401)
// resp.Write([]byte(`{"success": false}`))
// return
//}
body, err := ioutil.ReadAll(request.Body)
if err != nil {
@@ -817,16 +645,6 @@ func getDockerImage(resp http.ResponseWriter, request *http.Request) {
Name string `datastore:"name" json:"name" yaml:"name"`
}
//body = []byte(`swagger: "2.0"`)
//body = []byte(`swagger: '1.0'`)
//newbody := string(body)
//newbody = strings.TrimSpace(newbody)
//body = []byte(newbody)
//log.Println(string(body))
//tmpbody, err := yaml.YAMLToJSON(body)
//log.Println(err)
//log.Println(string(tmpbody))
// This has to be done in a weird way because Datastore doesn't
// support map[string]interface and similar (openapi3.Swagger)
var version requestCheck
@@ -838,16 +656,10 @@ func getDockerImage(resp http.ResponseWriter, request *http.Request) {
return
}
log.Printf("Image to load: %s", version.Name)
//cli, err := client.NewEnvClient()
//if err != nil {
// log.Println("Unable to create docker client")
// return err
//}
//log.Printf("[DEBUG] Image to load: %s", version.Name)
dockercli, err := client.NewEnvClient()
if err != nil {
log.Printf("Unable to create docker client: %s", err)
log.Printf("[WARNING] Unable to create docker client: %s", err)
resp.WriteHeader(422)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed JSON marshalling: %s"}`, err)))
return
@@ -860,37 +672,241 @@ func getDockerImage(resp http.ResponseWriter, request *http.Request) {
img := types.ImageSummary{}
tagFound := ""
img2 := types.ImageSummary{}
tagFound2 := ""
alternativeNameSplit := strings.Split(version.Name, "/")
alternativeName := version.Name
if len(alternativeNameSplit) == 3 {
alternativeName = strings.Join(alternativeNameSplit[1:3], "/")
}
for _, image := range images {
for _, tag := range image.RepoTags {
log.Printf("Image: %s", tag)
if strings.ToLower(tag) == strings.ToLower(version.Name) {
img = image
tagFound = tag
break
}
if strings.ToLower(tag) == strings.ToLower(alternativeName) {
img2 = image
tagFound2 = tag
}
}
}
// REBUILDS THE APP
if len(img.ID) == 0 {
if len(img2.ID) == 0 {
workflowapps, err := shuffle.GetAllWorkflowApps(ctx, 0, 0)
//log.Printf("[INFO] Getting workflowapps for a rebuild. Got %d with err %#v", len(workflowapps), err)
if err == nil {
imageName := ""
imageVersion := ""
newNameSplit := strings.Split(version.Name, ":")
if len(newNameSplit) == 2 {
//log.Printf("[DEBUG] Found name %#v", newNameSplit)
findVersionSplit := strings.Split(newNameSplit[1], "_")
//log.Printf("[DEBUG] Found another split %#v", findVersionSplit)
if len(findVersionSplit) == 2 {
imageVersion = findVersionSplit[len(findVersionSplit)-1]
imageName = findVersionSplit[0]
} else if len(findVersionSplit) >= 2 {
imageVersion = findVersionSplit[len(findVersionSplit)-1]
imageName = strings.Join(findVersionSplit[0:len(findVersionSplit)-1], "_")
} else {
log.Printf("[DEBUG] Couldn't parse appname & version for %#v", findVersionSplit)
}
}
if len(imageName) > 0 && len(imageVersion) > 0 {
foundApp := shuffle.WorkflowApp{}
imageName = strings.ToLower(imageName)
imageVersion = strings.ToLower(imageVersion)
log.Printf("[DEBUG] Looking for appname %s with version %s", imageName, imageVersion)
for _, app := range workflowapps {
if strings.ToLower(strings.Replace(app.Name, " ", "_", -1)) == imageName && app.AppVersion == imageVersion {
if app.Generated {
log.Printf("[DEBUG] Found matching app %s:%s - %s", imageName, imageVersion, app.ID)
foundApp = app
break
} else {
log.Printf("[WARNING] Trying to rebuild app that isn't generated - not allowed. Looking further.")
}
//break
}
}
if len(foundApp.ID) > 0 {
openApiApp, err := shuffle.GetOpenApiDatastore(ctx, foundApp.ID)
if err != nil {
log.Printf("[ERROR] Failed getting OpenAPI app %s to database: %s", foundApp.ID, err)
} else {
log.Printf("[DEBUG] Found OpenAPI app for %s as generated - now building!", version.Name)
user := shuffle.User{}
//img = version.Name
if len(alternativeName) > 0 {
tagFound = alternativeName
} else {
tagFound = version.Name
}
buildSwaggerApp(resp, []byte(openApiApp.Body), user)
}
}
}
} else {
log.Printf("[WARNING] Couldn't find an image with registry name %s and %s", version.Name, alternativeName)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "Couldn't find image %s"}`, version.Name)))
return
}
}
if len(tagFound) == 0 && len(tagFound2) > 0 {
img = img2
tagFound = tagFound2
}
}
//log.Printf("[INFO] Img found (%s): %#v", tagFound, img)
//log.Printf("[INFO] Img found to be downloaded by client: %s", tagFound)
newClient, err := newdockerclient.NewClientFromEnv()
if err != nil {
log.Printf("[ERROR] Failed setting up docker env: %#v", newClient)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "Couldn't find image %s"}`, version.Name)))
resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "Couldn't make docker client"}`)))
return
}
_ = tagFound
/*
log.Printf("IMg: %#v", img)
pullOptions := types.ImagePullOptions{}
log.Printf("[INFO] Pulling image %s", image)
reader, err := dockercli.ImagePull(ctx, tag, pullOptions)
if err != nil {
log.Printf("[ERROR] Failed getting image %s: %s", image, err)
////https://github.com/fsouza/go-dockerclient/issues/600
//defer fileReader.Close()
opts := newdockerclient.ExportImageOptions{
Name: tagFound,
OutputStream: resp,
}
if err := newClient.ExportImage(opts); err != nil {
log.Printf("[ERROR] FAILED to save image to file: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "Couldn't export image"}`)))
return
}
//resp.WriteHeader(200)
}
func activateWorkflowAppDocker(resp http.ResponseWriter, request *http.Request) {
cors := shuffle.HandleCors(resp, request)
if cors {
return
}
user, err := shuffle.HandleApiAuthentication(resp, request)
if err != nil {
log.Printf("[WARNING] Api authentication failed in get active apps: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
if user.Role == "org-reader" {
log.Printf("[WARNING] Org-reader doesn't have access to activate workflow app (shared): %s (%s)", user.Username, user.Id)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Read only user"}`))
return
}
ctx := context.Background()
location := strings.Split(request.URL.String(), "/")
var fileId string
if location[1] == "api" {
if len(location) <= 4 {
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
io.Copy(os.Stdout, r)
*/
fileId = location[4]
}
app, err := shuffle.GetApp(ctx, fileId, user, false)
if err != nil {
appName := request.URL.Query().Get("app_name")
appVersion := request.URL.Query().Get("app_version")
if len(appName) > 0 && len(appVersion) > 0 {
apps, err := shuffle.FindWorkflowAppByName(ctx, appName)
//log.Printf("[INFO] Found %d apps for %s", len(apps), appName)
if err != nil || len(apps) == 0 {
log.Printf("[WARNING] Error getting app %s (app config): %s", appName, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "App doesn't exist"}`))
return
}
selectedApp := shuffle.WorkflowApp{}
for _, app := range apps {
if !app.Sharing && !app.Public {
continue
}
if app.Name == appName {
selectedApp = app
}
if app.Name == appName && app.AppVersion == appVersion {
selectedApp = app
}
}
app = &selectedApp
} else {
log.Printf("[WARNING] Error getting app with ID %s (app config): %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "App doesn't exist"}`))
return
}
}
if app.Sharing || app.Public {
org, err := shuffle.GetOrg(ctx, user.ActiveOrg.Id)
if err == nil {
added := false
if !shuffle.ArrayContains(org.ActiveApps, app.ID) {
org.ActiveApps = append(org.ActiveApps, app.ID)
added = true
}
if added {
err = shuffle.SetOrg(ctx, *org, org.Id)
if err != nil {
log.Printf("[WARNING] Failed setting org when autoadding apps on save: %s", err)
} else {
log.Printf("[INFO] Added public app %s (%s) to org %s (%s)", app.Name, app.ID, user.ActiveOrg.Name, user.ActiveOrg.Id)
cacheKey := fmt.Sprintf("apps_%s", user.Id)
shuffle.DeleteCache(ctx, cacheKey)
}
}
}
} else {
log.Printf("[WARNING] User is trying to activate %s which is NOT public", app.Name)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
log.Printf("[DEBUG] App %s (%s) activated for org %s by user %s", app.Name, app.ID, user.ActiveOrg.Id, user.Username)
// If onprem, it should autobuild the container(s) from here
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"success": true, "message": "Downloading image %s"}`, version.Name)))
resp.Write([]byte(`{"success": true}`))
}
backend/go-app/files.go
-873
View File
@@ -1,873 +0,0 @@
package main
/*
Handles files within Workflows.of Shuffle
*/
import (
"bytes"
"context"
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"strconv"
"strings"
"time"
"cloud.google.com/go/datastore"
"github.com/satori/go.uuid"
)
type File struct {
Id string `json:"id" datastore:"id"`
Type string `json:"type" datastore:"type"`
CreatedAt int64 `json:"created_at" datastore:"created_at"`
UpdatedAt int64 `json:"updated_at" datastore:"updated_at"`
MetaAccessAt int64 `json:"meta_access_at" datastore:"meta_access_at"`
DownloadAt int64 `json:"last_downloaded" datastore:"last_downloaded"`
Description string `json:"description" datastore:"description"`
ExpiresAt string `json:"expires_at" datastore:"expires_at"`
Status string `json:"status" datastore:"status"`
Filename string `json:"filename" datastore:"filename"`
URL string `json:"url" datastore:"org"`
OrgId string `json:"org_id" datastore:"org_id"`
WorkflowId string `json:"workflow_id" datastore:"workflow_id"`
Workflows []string `json:"workflows" datastore:"workflows"`
DownloadPath string `json:"download_path" datastore:"download_path"`
Md5sum string `json:"md5_sum" datastore:"md5_sum"`
Sha256sum string `json:"sha256_sum" datastore:"sha256_sum"`
FileSize int64 `json:"filesize" datastore:"filesize"`
Duplicate bool `json:"duplicate" datastore:"duplicate"`
Subflows []string `json:"subflows" datastore:"subflows"`
}
var basepath = os.Getenv("SHUFFLE_FILE_LOCATION")
func fileAuthentication(request *http.Request) (string, error) {
executionId, ok := request.URL.Query()["execution_id"]
if ok && len(executionId) > 0 {
ctx := context.Background()
workflowExecution, err := getWorkflowExecution(ctx, executionId[0])
if err != nil {
log.Printf("[ERROR] Couldn't find execution ID %s", executionId[0])
return "", err
}
apikey := request.Header.Get("Authorization")
if !strings.HasPrefix(apikey, "Bearer ") {
log.Printf("[ERROR} Apikey doesn't start with bearer (2)")
return "", errors.New("No auth key found")
}
apikeyCheck := strings.Split(apikey, " ")
if len(apikeyCheck) != 2 {
log.Printf("[ERROR] Invalid format for apikey (2)")
return "", errors.New("No space in authkey")
}
// This is annoying af and is done because of maxlength lol
newApikey := apikeyCheck[1]
if newApikey != workflowExecution.Authorization {
//log.Printf("[ERROR] Bad apikey for execution %s. %s vs %s", executionId[0], apikey, workflowExecution.Authorization)
log.Printf("[ERROR] Bad apikey for execution %s.", executionId[0])
//%s vs %s", executionId[0], apikey, workflowExecution.Authorization)
return "", errors.New("Bad authorization key")
}
log.Printf("[INFO] Authorization is correct for execution %s!", executionId[0])
//%s vs %s. Setting Org", executionId, apikey, workflowExecution.Authorization)
if len(workflowExecution.ExecutionOrg) > 0 {
return workflowExecution.ExecutionOrg, nil
} else if len(workflowExecution.Workflow.ExecutingOrg.Id) > 0 {
return workflowExecution.ExecutionOrg, nil
} else {
log.Printf("[ERROR] Couldn't find org for workflow execution, but auth was correct.")
}
}
return "", errors.New("No execution id specified")
}
// https://golangcode.com/check-if-a-file-exists/
func fileExists(filename string) bool {
info, err := os.Stat(filename)
if os.IsNotExist(err) {
return false
}
return !info.IsDir()
}
func handleGetFiles(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("[INFO] INITIAL Api authentication failed in file LIST: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
if user.Role != "admin" {
log.Printf("[AUTH] User isn't admin")
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Need to be admin"}`)))
return
}
ctx := context.Background()
files, err := getAllFiles(ctx, user.ActiveOrg.Id)
if err != nil {
log.Printf("[ERROR] Failed to get files: %s", err)
resp.WriteHeader(500)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Error getting files."}`)))
return
}
log.Printf("Got %d files for org %s", len(files), user.ActiveOrg.Id)
newBody, err := json.Marshal(files)
if err != nil {
log.Printf("[ERROR] Failed marshaling files: %s", err)
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false, "reason": "Failed to marshal files"}`))
return
}
resp.WriteHeader(200)
resp.Write([]byte(newBody))
}
func handleGetFileMeta(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
var fileId string
location := strings.Split(request.URL.String(), "/")
if location[1] == "api" {
if len(location) <= 4 {
log.Printf("[INFO] Path too short: %d", len(location))
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
fileId = location[4]
}
if strings.Contains(fileId, "?") {
fileId = strings.Split(fileId, "?")[0]
}
if len(fileId) != 36 {
log.Printf("Bad format for fileId %s", fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
return
}
log.Printf("\n\n[INFO] User is trying to GET File Meta for %s\n\n", fileId)
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("[INFO] INITIAL Api authentication failed in file deletion: %s", err)
orgId, err := fileAuthentication(request)
if err != nil {
log.Printf("[ERROR] Bad file authentication in get: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
user.ActiveOrg.Id = orgId
user.Username = "Execution File API"
}
// 1. Verify if the user has access to the file: org_id and workflow
log.Printf("[INFO] Should GET FILE META for %s if user has access", fileId)
ctx := context.Background()
file, err := getFile(ctx, fileId)
if err != nil {
log.Printf("[INFO] File %s not found: %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
found := false
if file.OrgId == user.ActiveOrg.Id {
found = true
} else {
for _, item := range user.Orgs {
if item == file.OrgId {
found = true
break
}
}
}
if !found {
log.Printf("[INFO] User %s doesn't have access to %s", user.Username, fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
newBody, err := json.Marshal(file)
if err != nil {
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false, "reason": "Failed to marshal filedata"}`))
return
}
log.Printf("[INFO] Successfully got file meta for %s", fileId)
resp.WriteHeader(200)
resp.Write([]byte(newBody))
}
func handleDeleteFile(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
var fileId string
location := strings.Split(request.URL.String(), "/")
if location[1] == "api" {
if len(location) <= 4 {
log.Printf("[INFO] Path too short: %d", len(location))
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
fileId = location[4]
}
if strings.Contains(fileId, "?") {
fileId = strings.Split(fileId, "?")[0]
}
if len(fileId) != 36 {
log.Printf("Bad format for fileId %s", fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
return
}
log.Printf("\n\n[INFO] User is trying to delete file %s\n\n", fileId)
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("[INFO] INITIAL Api authentication failed in file deletion: %s", err)
orgId, err := fileAuthentication(request)
if err != nil {
log.Printf("[ERROR] Bad file authentication in get: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
user.ActiveOrg.Id = orgId
user.Username = "Execution File API"
}
// 1. Verify if the user has access to the file: org_id and workflow
log.Printf("[INFO] Should DELETE file %s if user has access", fileId)
ctx := context.Background()
file, err := getFile(ctx, fileId)
if err != nil {
log.Printf("[INFO] File %s not found: %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
found := false
if file.OrgId == user.ActiveOrg.Id {
found = true
} else {
for _, item := range user.Orgs {
if item == file.OrgId {
found = true
break
}
}
}
if !found {
log.Printf("[INFO] User %s doesn't have access to %s", user.Username, fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
if file.Status == "deleted" {
log.Printf("[INFO] File with ID %s is already deleted.", fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
if fileExists(file.DownloadPath) {
err = os.Remove(file.DownloadPath)
if err != nil {
log.Printf("[ERROR] Failed deleting file locally: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed deleting filein path %s"}`, file.DownloadPath)))
return
}
log.Printf("[INFO] Deleted file %s locally. Next is database.", file.DownloadPath)
} else {
log.Printf("[ERROR] File doesn't exist. Can't delete. Should maybe delete file anyway?")
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "File in location %s doesn't exist"}`, file.DownloadPath)))
return
}
file.Status = "deleted"
err = setFile(ctx, *file)
if err != nil {
log.Printf("[ERROR] Failed setting file to deleted")
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false, "reason": "Failed setting file to deleted"}`))
return
}
/*
//Actually delete it?
err = DeleteKey(ctx, "files", fileId)
if err != nil {
log.Printf("Failed deleting file with ID %s: %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
*/
log.Printf("[INFO] Successfully deleted file %s", fileId)
resp.WriteHeader(200)
resp.Write([]byte(`{"success": true}`))
}
func handleGetFileContent(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
var fileId string
location := strings.Split(request.URL.String(), "/")
if location[1] == "api" {
if len(location) <= 4 {
log.Printf("Path too short: %d", len(location))
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
fileId = location[4]
}
if len(fileId) != 36 {
log.Printf("Bad format for fileId %s", fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
return
}
log.Printf("\n\n[INFO] User is trying to download file %s\n\n", fileId)
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("INITIAL Api authentication failed in file download: %s", err)
orgId, err := fileAuthentication(request)
if err != nil {
log.Printf("Bad file authentication in get: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
user.ActiveOrg.Id = orgId
user.Username = "Execution File API"
/*
} else {
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
*/
}
// 1. Verify if the user has access to the file: org_id and workflow
log.Printf("[INFO] Should get file %s", fileId)
ctx := context.Background()
file, err := getFile(ctx, fileId)
if err != nil {
log.Printf("[ERROR] File %s not found: %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
found := false
if file.OrgId == user.ActiveOrg.Id {
found = true
} else {
for _, item := range user.Orgs {
if item == file.OrgId {
found = true
break
}
}
}
if !found {
log.Printf("User %s doesn't have access to %s", user.Username, fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
if file.Status != "active" {
log.Printf("[ERROR] File status isn't active, but %s. Can't continue.", file.Status)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "The file isn't ready to be downloaded yet. Status required: active"}`))
return
}
// Fixme: More auth: org and workflow!
downloadPath := file.DownloadPath
log.Printf("[INFO] Downloadpath: %s", downloadPath)
Openfile, err := os.Open(downloadPath)
defer Openfile.Close() //Close after function return
if err != nil {
file.Status = "deleted"
err = setFile(ctx, *file)
if err != nil {
log.Printf("Failed setting file to uploading")
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false, "reason": "Failed setting file to uploading"}`))
return
}
//File not found, send 404
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "File doesn't exist locally"}`))
return
}
//File is found, create and send the correct headers
//Get the Content-Type of the file
//Create a buffer to store the header of the file in
FileHeader := make([]byte, 512)
//Copy the headers into the FileHeader buffer
Openfile.Read(FileHeader)
//Get content type of file
FileContentType := http.DetectContentType(FileHeader)
//Get the file size
FileStat, _ := Openfile.Stat() //Get info from file
FileSize := strconv.FormatInt(FileStat.Size(), 10) //Get file size as a string
//Send the headers
resp.Header().Set("Content-Disposition", "attachment; filename="+fileId)
resp.Header().Set("Content-Type", FileContentType)
resp.Header().Set("Content-Length", FileSize)
//Send the file
//We read 512 bytes from the file already, so we reset the offset back to 0
Openfile.Seek(0, 0)
io.Copy(resp, Openfile) //'Copy' the file to the client
return
//log.Printf("Should download file %s", downloadPath)
}
func handleUploadFile(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
var fileId string
location := strings.Split(request.URL.String(), "/")
if location[1] == "api" {
if len(location) <= 4 {
log.Printf("Path too short: %d", len(location))
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
fileId = location[4]
}
if len(fileId) != 36 {
log.Printf("Bad format for fileId %s", fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
return
}
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("INITIAL Api authentication failed in file upload: %s", err)
orgId, err := fileAuthentication(request)
if err != nil {
log.Printf("Bad file authentication in create file: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
user.ActiveOrg.Id = orgId
user.Username = "Execution File API"
}
log.Printf("[INFO] Should UPLOAD file %s if user has access", fileId)
ctx := context.Background()
file, err := getFile(ctx, fileId)
if err != nil {
log.Printf("File %s not found: %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
found := false
if file.OrgId == user.ActiveOrg.Id {
found = true
} else {
for _, item := range user.Orgs {
if item == file.OrgId {
found = true
break
}
}
}
if !found {
log.Printf("User %s doesn't have access to %s", user.Username, fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
log.Printf("[INFO] STATUS: %s", file.Status)
if file.Status != "created" {
log.Printf("File status isn't created. Can't upload.")
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "This file already has data."}`))
return
}
request.ParseMultipartForm(32 << 20)
parsedFile, _, err := request.FormFile("shuffle_file")
if err != nil {
log.Printf("[ERROR] Couldn't upload file: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Failed uploading file"}`))
return
}
defer parsedFile.Close()
file.Status = "uploading"
err = setFile(ctx, *file)
if err != nil {
log.Printf("Failed setting file to uploading")
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false, "reason": "Failed setting file to uploading"}`))
return
}
// Can be used for validation files for change
var buf bytes.Buffer
io.Copy(&buf, parsedFile)
contents := buf.Bytes()
file.FileSize = int64(len(contents))
md5 := md5sum(contents)
buf.Reset()
sha256Sum := sha256.Sum256(contents)
//parsedFile.Reset()
f, err := os.OpenFile(file.DownloadPath, os.O_WRONLY|os.O_CREATE, os.ModePerm)
if err != nil {
// Rolling back file
file.Status = "created"
setFile(ctx, *file)
log.Printf("[ERROR] Failed uploading and creating file: %s", err)
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false}`))
return
}
defer f.Close()
parsedFile.Seek(0, io.SeekStart)
io.Copy(f, parsedFile)
// FIXME: Set this one to 200 anyway? Can't download file then tho..
file.Status = "active"
file.Md5sum = md5
file.Sha256sum = fmt.Sprintf("%x", sha256Sum)
log.Printf("[INFO] MD5 for file %s (%s) is %s and SHA256 is %s", file.Filename, file.Id, file.Md5sum, file.Sha256sum)
err = setFile(ctx, *file)
if err != nil {
log.Printf("[ERROR] Failed setting file back to active")
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false, "reason": "Failed setting file to active"}`))
return
}
log.Printf("[INFO] Successfully uploaded file ID %s", file.Id)
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"success": true}`)))
}
func handleCreateFile(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("[INFO] INITIAL Api authentication failed in file creation: %s", err)
orgId, err := fileAuthentication(request)
if err != nil {
log.Printf("[ERROR] Bad file authentication in create file: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
user.ActiveOrg.Id = orgId
user.Username = "Execution File API"
}
body, err := ioutil.ReadAll(request.Body)
if err != nil {
log.Println("Failed reading body")
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed to read data"}`)))
return
}
type FileStructure struct {
Filename string `json:"filename"`
OrgId string `json:"org_id"`
WorkflowId string `json:"workflow_id"`
}
var curfile FileStructure
err = json.Unmarshal(body, &curfile)
if err != nil {
log.Printf("[ERROR] Failed unmarshaling: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed to unmarshal data"}`)))
return
}
// Loads of validation below
if len(curfile.Filename) == 0 || len(curfile.OrgId) == 0 || len(curfile.WorkflowId) == 0 {
log.Printf("[ERROR] Missing field during fileupload. Required: filename, org_id, workflow_id")
log.Printf("INPUT: %s", string(body))
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Missing field. Required: filename, org_id, workflow_id"}`)))
return
}
ctx := context.Background()
if user.ActiveOrg.Id != curfile.OrgId {
log.Printf("[ERROR] User can't access org %s", curfile.OrgId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Error with organization"}`))
return
}
// Try to get the org and workflow in case they don't exist
workflow, err := getWorkflow(ctx, curfile.WorkflowId)
if err != nil {
log.Printf("[ERROR] Workflow %s doesn't exist.", curfile.WorkflowId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Error with workflow id or org id"}`))
return
}
_, err = getOrg(ctx, curfile.OrgId)
if err != nil {
log.Printf("[ERROR] Org %s doesn't exist.", curfile.OrgId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Error with workflow id or org id"}`))
return
}
if workflow.ExecutingOrg.Id != curfile.OrgId {
found := false
for _, curorg := range workflow.Org {
if curorg.Id == curfile.OrgId {
found = true
break
}
}
if !found {
log.Printf("[ERROR] Org %s doesn't have access to %s.", curfile.OrgId, curfile.WorkflowId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Error with workflow id or org id"}`))
return
}
}
if strings.Contains(curfile.Filename, "/") || strings.Contains(curfile.Filename, `"`) || strings.Contains(curfile.Filename, "..") || strings.Contains(curfile.Filename, "~") {
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Invalid characters in filename"}`))
return
}
// 1. Create the file object.
if len(basepath) == 0 {
basepath = "shuffle-files"
}
folderPath := fmt.Sprintf("%s/%s/%s", basepath, curfile.OrgId, curfile.WorkflowId)
// Try to make the full file location
err = os.MkdirAll(folderPath, os.ModePerm)
if err != nil {
log.Printf("[ERROR] Writing issue for file location creation: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Failed creating upload location"}`))
return
}
filename := curfile.Filename
fileId := uuid.NewV4().String()
downloadPath := fmt.Sprintf("%s/%s", folderPath, fileId)
duplicateWorkflows := []string{}
for _, trigger := range workflow.Triggers {
if trigger.AppName == "Shuffle Workflow" && trigger.TriggerType == "SUBFLOW" {
for _, parameter := range trigger.Parameters {
if parameter.Name == "workflow" && len(parameter.Value) > 0 {
found := false
for _, workflow := range duplicateWorkflows {
if workflow == parameter.Value {
found = true
break
}
}
if !found {
duplicateWorkflows = append(duplicateWorkflows, parameter.Value)
}
break
}
}
}
}
timeNow := time.Now().Unix()
newFile := File{
Id: fileId,
CreatedAt: timeNow,
UpdatedAt: timeNow,
Description: "",
Status: "created",
Filename: filename,
OrgId: curfile.OrgId,
WorkflowId: curfile.WorkflowId,
DownloadPath: downloadPath,
Subflows: duplicateWorkflows,
}
err = setFile(ctx, newFile)
if err != nil {
log.Printf("[ERROR] Failed setting file: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Failed setting file reference"}`))
return
} else {
log.Printf("[INFO] Created file %s", newFile.DownloadPath)
}
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"success": true, "id": "%s"}`, fileId)))
}
func getFile(ctx context.Context, id string) (*File, error) {
key := datastore.NameKey("Files", id, nil)
curFile := &File{}
if err := dbclient.Get(ctx, key, curFile); err != nil {
return &File{}, err
}
return curFile, nil
}
func setFile(ctx context.Context, file File) error {
// clear session_token and API_token for user
timeNow := time.Now().Unix()
file.UpdatedAt = timeNow
k := datastore.NameKey("Files", file.Id, nil)
if _, err := dbclient.Put(ctx, k, &file); err != nil {
log.Println(err)
return err
}
return nil
}
func getAllFiles(ctx context.Context, orgId string) ([]File, error) {
var files []File
q := datastore.NewQuery("Files").Filter("org_id =", orgId).Order("-updated_at").Limit(100)
_, err := dbclient.GetAll(ctx, q, &files)
if err != nil {
if strings.Contains(fmt.Sprintf("%s", err), "ResourceExhausted") {
q = q.Limit(50)
_, err := dbclient.GetAll(ctx, q, &files)
if err != nil {
return []File{}, err
}
} else {
return []File{}, err
}
}
return files, nil
}
backend/go-app/go.mod
+26 -28
View File
@@ -1,37 +1,35 @@
module shuffle
module main
go 1.13
go 1.16
replace github.com/shuffle/shuffle-shared => ../../../shuffle-shared
//replace github.com/frikky/kin-openapi => ../../../../git/kin-openapi
//replace github.com/frikky/go-elasticsearch => ../../../../git/go-elasticsearch
require (
cloud.google.com/go v0.57.0
cloud.google.com/go/datastore v1.1.0
cloud.google.com/go/pubsub v1.3.1
cloud.google.com/go/storage v1.7.0
github.com/Microsoft/go-winio v0.4.14 // indirect
cloud.google.com/go/datastore v1.6.0
cloud.google.com/go/iam v0.1.1 // indirect
cloud.google.com/go/pubsub v1.17.1
cloud.google.com/go/storage v1.18.2
github.com/basgys/goxml2json v1.1.0
github.com/carlescere/scheduler v0.0.0-20170109141437-ee74d2f83d82
github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/docker/docker v1.13.1
github.com/docker/go-connections v0.4.0
github.com/docker/go-units v0.4.0 // indirect
github.com/getkin/kin-openapi v0.8.0
github.com/docker/docker v20.10.12+incompatible
github.com/frikky/kin-openapi v0.41.0
github.com/fsouza/go-dockerclient v1.7.7
github.com/ghodss/yaml v1.0.0
github.com/go-git/go-billy/v5 v5.0.0
github.com/go-git/go-git/v5 v5.0.0
github.com/google/go-github/v28 v28.1.1
github.com/gorilla/handlers v1.4.2 // indirect
github.com/gorilla/mux v1.7.4
github.com/h2non/filetype v1.0.12
github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/go-git/go-billy/v5 v5.3.1
github.com/go-git/go-git/v5 v5.4.2
github.com/gorilla/mux v1.8.0
github.com/h2non/filetype v1.1.3
github.com/nirasan/go-oauth-pkce-code-verifier v0.0.0-20170819232839-0fbfe93532da // indirect
github.com/satori/go.uuid v1.2.0
golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
google.golang.org/api v0.23.0
google.golang.org/appengine v1.6.6
google.golang.org/genproto v0.0.0-20200507105951-43844f6eee31
google.golang.org/grpc v1.29.1
github.com/shuffle/shuffle-shared v0.2.29
go4.org v0.0.0-20201209231011-d4a079459e60 // indirect
golang.org/x/crypto v0.0.0-20220112180741-5e0467b6c7ce
google.golang.org/api v0.65.0
google.golang.org/appengine v1.6.7
google.golang.org/grpc v1.43.0
gopkg.in/src-d/go-git.v4 v4.13.1
gopkg.in/yaml.v2 v2.2.8
gopkg.in/yaml.v3 v3.0.0-20200506231410-2ff61e1afc86
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
)
backend/go-app/go.sum
-435
View File
@@ -1,435 +0,0 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=
cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=
cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
cloud.google.com/go v0.57.0 h1:EpMNVUorLiZIELdMZbCYX/ByTFCdoYopYAGxaGVz9ms=
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
cloud.google.com/go/bigquery v1.6.0/go.mod h1:hyFDG0qSGdHNz8Q6nDN8rYIkld0q/+5uBZaelxiDLfE=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0 h1:/May9ojXjRkPBNVrq+oWLqmWCkr4OU5uRY29bu0mRyQ=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
cloud.google.com/go/pubsub v1.3.1 h1:ukjixP1wl0LpnZ6LWtZJ0mX5tBmjp1f8Sqer8Z2OMUU=
cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=
cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
cloud.google.com/go/storage v1.7.0 h1:DzdLPI8Em+DEk7IzA2a10ivq3mxIEASC9GeNJ6FFt5Q=
cloud.google.com/go/storage v1.7.0/go.mod h1:jGMIBwF+L/tL6WN/W5InNgYYu4HP0DvGB6rQ1mufWfs=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/Microsoft/go-winio v0.4.14 h1:+hMXMk01us9KgxGb7ftKQt2Xpf5hH/yky+TDA+qxleU=
github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA=
github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7/go.mod h1:6zEj6s6u/ghQa61ZWa/C2Aw3RkjiTBOix7dkqa1VLIs=
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
github.com/basgys/goxml2json v1.1.0 h1:4ln5i4rseYfXNd86lGEB+Vi652IsIXIvggKM/BhUKVw=
github.com/basgys/goxml2json v1.1.0/go.mod h1:wH7a5Np/Q4QoECFIU8zTQlZwZkrilY0itPfecMw41Dw=
github.com/carlescere/scheduler v0.0.0-20170109141437-ee74d2f83d82 h1:9bAydALqAjBfPHd/eAiJBHnMZUYov8m2PkXVr+YGQeI=
github.com/carlescere/scheduler v0.0.0-20170109141437-ee74d2f83d82/go.mod h1:tyA14J0sA3Hph4dt+AfCjPrYR13+vVodshQSM7km9qw=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug=
github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v1.13.1 h1:IkZjBSIc8hBjLpqeAbeE5mca5mNgeatLHBy3GO78BWo=
github.com/docker/docker v1.13.1/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-units v0.4.0 h1:3uh0PgVws3nIA0Q+MwDC8yjEPf9zjRfZZWXZYDct3Tw=
github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/emirpasic/gods v1.12.0 h1:QAUIPSaCu4G+POclxeqb3F+WPpdKqFGlw36+yOzGlrg=
github.com/emirpasic/gods v1.12.0/go.mod h1:YfzfFFoVP/catgzJb4IKIqXjX78Ha8FMSDh3ymbK86o=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
github.com/getkin/kin-openapi v0.8.0 h1:a6TQjTqwkyscC4/hShJX7WhCVE+4bi9lzw61XHQW5hE=
github.com/getkin/kin-openapi v0.8.0/go.mod h1:zZQMFkVgRHCdhgb6ihCTIo9dyDZFvX0k/xAKqw1FhPw=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gliderlabs/ssh v0.2.2/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0=
github.com/go-git/gcfg v1.5.0 h1:Q5ViNfGF8zFgyJWPqYwA7qGFoMTEiBmdlkcfRmpIMa4=
github.com/go-git/gcfg v1.5.0/go.mod h1:5m20vg6GwYabIxaOonVkTdrILxQMpEShl1xiMF4ua+E=
github.com/go-git/go-billy v4.2.0+incompatible h1:Z6QtVXd5tjxUtcODLugkJg4WaZnGg13CD8qB9pr+7q0=
github.com/go-git/go-billy/v5 v5.0.0 h1:7NQHvd9FVid8VL4qVUMm8XifBK+2xCoZ2lSk0agRrHM=
github.com/go-git/go-billy/v5 v5.0.0/go.mod h1:pmpqyWchKfYfrkb/UVH4otLvyi/5gJlGI4Hb3ZqZ3W0=
github.com/go-git/go-git v4.7.0+incompatible h1:+W9rgGY4DOKKdX2x6HxSR7HNeTxqiKrOvKnuittYVdA=
github.com/go-git/go-git-fixtures/v4 v4.0.1/go.mod h1:m+ICp2rF3jDhFgEZ/8yziagdT1C+ZpZcrJjappBCDSw=
github.com/go-git/go-git/v5 v5.0.0 h1:k5RWPm4iJwYtfWoxIJy4wJX9ON7ihPeZZYC1fLYDnpg=
github.com/go-git/go-git/v5 v5.0.0/go.mod h1:oYD8y9kWsGINPFJoLdaScGCN6dlKg23blmClfZwtUVA=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0 h1:oOuy+ugB+P/kBdUnG5QaMXSIyJ1q38wWSojYCb3z5VQ=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0 h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4rEjNlfyDHW9dolSY=
github.com/google/go-github/v28 v28.1.1 h1:kORf5ekX5qwXO2mGzXXOjMe/g6ap8ahVe0sBEulhSxo=
github.com/google/go-github/v28 v28.1.1/go.mod h1:bsqJWQX05omyWVmc00nEUql9mhQyv38lDZ8kPZcQVoM=
github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5 h1:sjZBwGj9Jlw33ImPtvFviGYvseOtDM7hkSKB7+Tv3SM=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/gorilla/handlers v1.4.2 h1:0QniY0USkHQ1RGCLfKxeNHK9bkDHGRYGNDFBCS+YARg=
github.com/gorilla/handlers v1.4.2/go.mod h1:Qkdc/uu4tH4g6mTK6auzZ766c4CA0Ng8+o/OAirnOIQ=
github.com/gorilla/mux v1.7.4 h1:VuZ8uybHlWmqV03+zRzdwKL4tUnIp1MAQtp1mIFE1bc=
github.com/gorilla/mux v1.7.4/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/h2non/filetype v1.0.12 h1:yHCsIe0y2cvbDARtJhGBTD2ecvqMSTvlIcph9En/Zao=
github.com/h2non/filetype v1.0.12/go.mod h1:319b3zT68BvV+WRj7cwy856M2ehB3HqNOt6sy1HndBY=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1 h1:6QPYqodiu3GuPL+7mfx+NwDdp2eTkp9IfEUpgAwUN0o=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/kevinburke/ssh_config v0.0.0-20190725054713-01f96b0aa0cd h1:Coekwdh0v2wtGp9Gmz1Ze3eVRAWJMLokvN3QjdzCHLY=
github.com/kevinburke/ssh_config v0.0.0-20190725054713-01f96b0aa0cd/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF4nAY/ojJ6r6mM=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.8/go.mod h1:O1sed60cT9XZ5uDucP5qwvh+TE3NnUj51EiZO/lmSfw=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/opencontainers/go-digest v1.0.0-rc1 h1:WzifXhOVOEOuFYOJAW6aQqW0TooG2iki3E3Ii+WN7gQ=
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pelletier/go-buffruneio v0.2.0/go.mod h1:JkE26KsDizTr40EUHkXVtNPvgGtbSNq5BcowyYOWdKo=
github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/satori/go.uuid v1.2.0 h1:0uYX9dsZ2yD7q2RtLRtPSdGDWzjeM3TbMJP9utgA0ww=
github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/src-d/gcfg v1.4.0 h1:xXbNR5AlLSA315x2UO+fTSSAXCDf+Ar38/6oyGbDKQ4=
github.com/src-d/gcfg v1.4.0/go.mod h1:p/UMsR43ujA89BJY9duynAwIpvqEujIH/jFlfL7jWoI=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/xanzy/ssh-agent v0.2.1 h1:TCbipTQL2JiiCprBWx9frJ2eJlCYT00NmctrHxVAr70=
github.com/xanzy/ssh-agent v0.2.1/go.mod h1:mLlQY/MoOhWBj+gOGMQkOeiEvkx+8pJSI+0Bx9h2kr4=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3 h1:8sGtKOrtQqkN1bp2AtX+misvLIlOmsEsNd+9NIcPEm8=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
golang.org/x/crypto v0.0.0-20190219172222-a4c6cb3142f2/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79 h1:IaQbIIB2X/Mp/DKctl6ROxz1KyMlKp4uyvL6+kQ7C88=
golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b h1:Wh+f8QHJXR411sJR8/vRBTZ7YapZaRvUcLFFJhusH0k=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0 h1:KU7oHjnv3XNWfa5COkzUifxZmxp1TyI7ImMXqFxLwvQ=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5 h1:WQ8q63x+f/zpC8Ac1s9wLElVoHhm32p6tudrU72n1QA=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a h1:WXEvlFVvvGxCJLG6REjsT03iWnKLEWinaScsxF2Vm2o=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190221075227-b4e8571b14e0/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200409092240-59c9f1ba88fa/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e h1:hq86ru83GdWTlfQFZGO4nZJTU4Bs2wfHl8oFHRaXsfc=
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190729092621-ff9f1409240a/go.mod h1:jcCCGcm9btYwXyDqrUWc6MKQKKGJCWEQ3AfLSRIbEuI=
golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
golang.org/x/tools v0.0.0-20200409170454-77362c5149f0/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d h1:lzLdP95xJmMpwQ6LUHwrc5V7js93hTiY7gkznu0BgmY=
golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.21.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.23.0 h1:YlvGEOq2NA2my8cZ/9V8BcEO9okD48FlJcdqN0xJL3s=
google.golang.org/api v0.23.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.6 h1:lMO5rYAqUxkmaj76jAkRUvt5JZgFymx/+Q5Mzfivuhc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200409111301-baae70f3302d/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200507105951-43844f6eee31 h1:Bz1qTn2YRWV+9OKJtxHJiQKCiXIdf+kwuKXdt9cBxyU=
google.golang.org/genproto v0.0.0-20200507105951-43844f6eee31/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
google.golang.org/grpc v1.28.1/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
google.golang.org/grpc v1.29.1 h1:EC2SB8S04d2r73uptxphDSUG+kTKVgjRPF+N3xpxRB4=
google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0 h1:qdOKuR/EIArgaWNjetjgTzgVTAZ+S/WXVrq9HW9zimw=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/src-d/go-billy.v4 v4.3.2 h1:0SQA1pRztfTFx2miS8sA97XvooFeNOmvUenF4o0EcVg=
gopkg.in/src-d/go-billy.v4 v4.3.2/go.mod h1:nDjArDMp+XMs1aFAESLRjfGSgfvoYN0hDfzEk0GjC98=
gopkg.in/src-d/go-git-fixtures.v3 v3.5.0/go.mod h1:dLBcvytrw/TYZsNTWCnkNF2DSIlzWYqTe3rJR56Ac7g=
gopkg.in/src-d/go-git.v4 v4.13.1 h1:SRtFyV8Kxc0UP7aCHcijOMQGPxHSmMOPrzulQWolkYE=
gopkg.in/src-d/go-git.v4 v4.13.1/go.mod h1:nx5NYcxdKxq5fpltdHnPa2Exj4Sx0EclMWZQbYDu2z8=
gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME=
gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200506231410-2ff61e1afc86 h1:OfFoIUYv/me30yv7XlMy4F9RJw8DEm8WQ6QG1Ph4bH0=
gopkg.in/yaml.v3 v3.0.0-20200506231410-2ff61e1afc86/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2020.1.3 h1:sXmLre5bzIR6ypkjXCDI3jHPssRhc8KD/Ome589sc3U=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
backend/go-app/main.go
+2058 -4419
View File
File diff suppressed because it is too large Load Diff
backend/go-app/main_test.go
+340
View File
@@ -0,0 +1,340 @@
package main
import (
"github.com/shuffle/shuffle-shared"
"bytes"
"context"
"log"
"net/http"
"net/http/httptest"
"reflect"
"runtime"
"testing"
"time"
"cloud.google.com/go/datastore"
"cloud.google.com/go/storage"
"google.golang.org/api/option"
"google.golang.org/grpc"
)
type endpoint struct {
handler http.HandlerFunc
path string
method string
}
func init() {
ctx := context.Background()
dbclient, err := datastore.NewClient(ctx, gceProject, option.WithGRPCDialOption(grpc.WithNoProxy()))
if err != nil {
log.Fatalf("[DEBUG] Database client error during init: %s", err)
}
_, err = shuffle.RunInit(*dbclient, storage.Client{}, gceProject, "onprem", true, "elasticsearch")
log.Printf("INIT")
}
// TestTestAuthenticationRequired tests that the handlers in the `handlers`
// variable returns 401 Unauthorized when called without credentials.
func TestAuthenticationRequired(t *testing.T) {
handlers := []endpoint{
{handler: shuffle.HandleNewOutlookRegister, path: "/functions/outlook/register", method: "GET"},
{handler: shuffle.HandleGetOutlookFolders, path: "/functions/outlook/getFolders", method: "GET"},
{handler: shuffle.HandleApiGeneration, path: "/api/v1/users/generateapikey", method: "GET"},
{handler: handleLogin, path: "/api/v1/users/login", method: "POST"}, // prob not this one
// handleRegister generates nil pointer exception. Not necessary for this anyway.
//{handler: handleRegister, path: "/api/v1/users/register", method: "POST"},
{handler: shuffle.HandleGetUsers, path: "/api/v1/users/getusers", method: "GET"},
{handler: handleInfo, path: "/api/v1/users/getinfo", method: "GET"},
{handler: shuffle.HandleSettings, path: "/api/v1/users/getsettings", method: "GET"},
{handler: shuffle.HandleUpdateUser, path: "/api/v1/users/updateuser", method: "PUT"},
{handler: shuffle.DeleteUser, path: "/api/v1/users/123", method: "DELETE"},
{handler: shuffle.HandlePasswordChange, path: "/api/v1/users/passwordchange", method: "POST"},
{handler: shuffle.HandleGetUsers, path: "/api/v1/users", method: "GET"},
{handler: shuffle.HandleGetEnvironments, path: "/api/v1/getenvironments", method: "GET"},
{handler: shuffle.HandleSetEnvironments, path: "/api/v1/setenvironments", method: "PUT"},
// handleWorkflowQueue generates nil pointer exception
//{handler: handleWorkflowQueue, path: "/api/v1/streams", method: "POST"},
// handleGetStreamResults generates nil pointer exception
//{handler: handleGetStreamResults, path: "/api/v1/streams/results", method: "POST"},
{handler: handleAppHotloadRequest, path: "/api/v1/apps/run_hotload", method: "GET"},
{handler: LoadSpecificApps, path: "/api/v1/apps/get_existing", method: "POST"},
{handler: shuffle.UpdateWorkflowAppConfig, path: "/api/v1/apps/123", method: "PATCH"},
{handler: validateAppInput, path: "/api/v1/apps/validate", method: "POST"},
{handler: shuffle.DeleteWorkflowApp, path: "/api/v1/apps/123", method: "DELETE"},
{handler: shuffle.GetWorkflowAppConfig, path: "/api/v1/apps/123/config", method: "GET"},
{handler: getWorkflowApps, path: "/api/v1/apps", method: "GET"},
{handler: setNewWorkflowApp, path: "/api/v1/apps", method: "PUT"},
//{handler: shuffle.GetSpecificApps, path: "/api/v1/apps/search", method: "POST"},
{handler: shuffle.GetAppAuthentication, path: "/api/v1/apps/authentication", method: "GET"},
{handler: shuffle.AddAppAuthentication, path: "/api/v1/apps/authentication", method: "PUT"},
{handler: shuffle.DeleteAppAuthentication, path: "/api/v1/apps/authentication/123", method: "DELETE"},
{handler: validateAppInput, path: "/api/v1/workflows/apps/validate", method: "POST"},
{handler: getWorkflowApps, path: "/api/v1/workflows/apps", method: "GET"},
{handler: setNewWorkflowApp, path: "/api/v1/workflows/apps", method: "PUT"},
{handler: shuffle.GetWorkflows, path: "/api/v1/workflows", method: "GET"},
{handler: shuffle.SetNewWorkflow, path: "/api/v1/workflows", method: "POST"},
{handler: handleGetWorkflowqueue, path: "/api/v1/workflows/queue", method: "GET"},
{handler: handleGetWorkflowqueueConfirm, path: "/api/v1/workflows/queue/confirm", method: "POST"},
{handler: shuffle.HandleGetSchedules, path: "/api/v1/workflows/schedules", method: "GET"},
{handler: loadSpecificWorkflows, path: "/api/v1/workflows/download_remote", method: "POST"},
{handler: executeWorkflow, path: "/api/v1/workflows/123/execute", method: "GET"},
{handler: scheduleWorkflow, path: "/api/v1/workflows/123/schedule", method: "POST"},
{handler: stopSchedule, path: "/api/v1/workflows/123/schedule/abc", method: "DELETE"},
// createOutlookSub generates nil pointer exception
{handler: shuffle.HandleCreateOutlookSub, path: "/api/v1/workflows/123/outlook", method: "POST"},
// handleDeleteOutlookSub generates nil pointer exception
{handler: shuffle.HandleDeleteOutlookSub, path: "/api/v1/workflows/123/outlook/abc", method: "DELETE"},
{handler: shuffle.GetWorkflowExecutions, path: "/api/v1/workflows/123/executions", method: "GET"},
{handler: shuffle.AbortExecution, path: "/api/v1/workflows/123/executions/abc/abort", method: "GET"},
{handler: shuffle.GetSpecificWorkflow, path: "/api/v1/workflows/123", method: "GET"},
{handler: shuffle.SaveWorkflow, path: "/api/v1/workflows/123", method: "PUT"},
{handler: deleteWorkflow, path: "/api/v1/workflows/123", method: "DELETE"},
{handler: shuffle.HandleNewHook, path: "/api/v1/hooks/new", method: "POST"},
{handler: handleWebhookCallback, path: "/api/v1/hooks/123", method: "POST"},
{handler: shuffle.HandleDeleteHook, path: "/api/v1/hooks/123/delete", method: "DELETE"},
{handler: shuffle.HandleGetSpecificTrigger, path: "/api/v1/triggers/123", method: "GET"},
//{handler: shuffle.HandleGetSpecificStats, path: "/api/v1/stats/123", method: "GET"},
{handler: verifySwagger, path: "/api/v1/verify_swagger", method: "POST"},
{handler: verifySwagger, path: "/api/v1/verify_openapi", method: "POST"},
{handler: echoOpenapiData, path: "/api/v1/get_openapi_uri", method: "POST"},
{handler: echoOpenapiData, path: "/api/v1/validate_openapi", method: "POST"},
{handler: shuffle.ValidateSwagger, path: "/api/v1/validate_openapi", method: "POST"},
{handler: getOpenapi, path: "/api/v1/get_openapi", method: "GET"},
//{handler: shuffle.CleanupExecutions, path: "/api/v1/execution_cleanup", method: "GET"},
{handler: handleCloudSetup, path: "/api/v1/cloud/setup", method: "POST"},
{handler: shuffle.HandleGetOrgs, path: "/api/v1/orgs", method: "POST"},
{handler: shuffle.HandleGetFileContent, path: "/api/v1/files/{fileId}/content", method: "POST"},
}
var err error
ctx := context.Background()
// Most handlers requires database access in order to not crash or cause
// nil pointer issues.
// To start a local database instance, run:
// docker-compose up database
// To let the tests know about the database, run:
// DATASTORE_EMULATOR_HOST=0.0.0.0:8000 go test
dbclient, err = datastore.NewClient(ctx, gceProject, option.WithGRPCDialOption(grpc.WithNoProxy()))
if err != nil {
t.Fatal(err)
}
dummyBody := bytes.NewBufferString("dummy")
for _, e := range handlers {
log.Printf("Endpoint: %#v", e.path)
req, err := http.NewRequest(e.method, e.path, dummyBody)
if err != nil {
t.Fatal(err)
}
rr := httptest.NewRecorder()
handler := http.HandlerFunc(e.handler)
timeoutHandler := http.TimeoutHandler(handler, 2*time.Second, `Request Timeout.`)
timeoutHandler.ServeHTTP(rr, req)
funcName := getFunctionNameFromFunction(e.handler)
if status := rr.Code; status != http.StatusUnauthorized {
t.Errorf("%s handler returned wrong status code: got %v want %v",
funcName, status, http.StatusUnauthorized)
}
}
}
func TestAuthenticationNotRequired(t *testing.T) {
// All of these return 200 OK when user not logged in
handlers := []endpoint{
{handler: checkAdminLogin, path: "/api/v1/users/checkusers", method: "GET"},
{handler: shuffle.HandleLogout, path: "/api/v1/users/logout", method: "POST"},
{handler: shuffle.GetDocList, path: "/api/v1/docs", method: "GET"},
{handler: shuffle.GetDocs, path: "/api/v1/docs/123", method: "GET"},
{handler: healthCheckHandler, path: "/api/v1/_ah/health"},
}
for _, e := range handlers {
log.Printf("Endpoint: %#v", e.path)
req, err := http.NewRequest(e.method, e.path, nil)
if err != nil {
t.Fatal(err)
}
rr := httptest.NewRecorder()
handler := http.HandlerFunc(e.handler)
timeoutHandler := http.TimeoutHandler(handler, 2*time.Second, `Request Timeout.`)
timeoutHandler.ServeHTTP(rr, req)
funcName := getFunctionNameFromFunction(e.handler)
if status := rr.Code; status != http.StatusOK {
t.Errorf("%s handler returned wrong status code: got %v want %v",
funcName, status, http.StatusOK)
}
}
}
// TestCors tests that all endpoints returns the same CORS headers when hit
// with an OPTIONS type request.
// It feels very fragile to test headers like this, especially for the
// "Access-Control-Allow-Origin", but this test should be helpful while
// refactoring the CORS logic into a middleware, and that's the reason this
// test exists right now. It might change after the refactor because our
// requirements might change after the refactor.
func TestCors(t *testing.T) {
handlers := []endpoint{
{handler: shuffle.HandleNewOutlookRegister, path: "/functions/outlook/register", method: "GET"},
{handler: shuffle.HandleGetOutlookFolders, path: "/functions/outlook/getFolders", method: "GET"},
{handler: shuffle.HandleApiGeneration, path: "/api/v1/users/generateapikey", method: "GET"},
{handler: handleLogin, path: "/api/v1/users/login", method: "POST"}, // prob not this one
// handleRegister generates nil pointer exception
{handler: handleRegister, path: "/api/v1/users/register", method: "POST"},
{handler: shuffle.HandleGetUsers, path: "/api/v1/users/getusers", method: "GET"},
{handler: handleInfo, path: "/api/v1/users/getinfo", method: "GET"},
{handler: shuffle.HandleSettings, path: "/api/v1/users/getsettings", method: "GET"},
{handler: shuffle.HandleUpdateUser, path: "/api/v1/users/updateuser", method: "PUT"},
{handler: shuffle.DeleteUser, path: "/api/v1/users/123", method: "DELETE"},
// handlePasswordChange generates nil pointer exception
{handler: shuffle.HandlePasswordChange, path: "/api/v1/users/passwordchange", method: "POST"},
{handler: shuffle.HandleGetUsers, path: "/api/v1/users", method: "GET"},
{handler: shuffle.HandleGetEnvironments, path: "/api/v1/getenvironments", method: "GET"},
{handler: shuffle.HandleSetEnvironments, path: "/api/v1/setenvironments", method: "PUT"},
// handleWorkflowQueue generates nil pointer exception
{handler: handleWorkflowQueue, path: "/api/v1/streams", method: "POST"},
// handleGetStreamResults generates nil pointer exception
{handler: handleGetStreamResults, path: "/api/v1/streams/results", method: "POST"},
{handler: handleAppHotloadRequest, path: "/api/v1/apps/run_hotload", method: "GET"},
{handler: LoadSpecificApps, path: "/api/v1/apps/get_existing", method: "POST"},
{handler: shuffle.UpdateWorkflowAppConfig, path: "/api/v1/apps/123", method: "PATCH"},
{handler: validateAppInput, path: "/api/v1/apps/validate", method: "POST"},
{handler: shuffle.DeleteWorkflowApp, path: "/api/v1/apps/123", method: "DELETE"},
{handler: shuffle.GetWorkflowAppConfig, path: "/api/v1/apps/123/config", method: "GET"},
{handler: getWorkflowApps, path: "/api/v1/apps", method: "GET"},
{handler: setNewWorkflowApp, path: "/api/v1/apps", method: "PUT"},
//{handler: shuffle.GetSpecificApps, path: "/api/v1/apps/search", method: "POST"},
{handler: shuffle.GetAppAuthentication, path: "/api/v1/apps/authentication", method: "GET"},
{handler: shuffle.AddAppAuthentication, path: "/api/v1/apps/authentication", method: "PUT"},
{handler: shuffle.DeleteAppAuthentication, path: "/api/v1/apps/authentication/123", method: "DELETE"},
{handler: validateAppInput, path: "/api/v1/workflows/apps/validate", method: "POST"},
{handler: getWorkflowApps, path: "/api/v1/workflows/apps", method: "GET"},
{handler: setNewWorkflowApp, path: "/api/v1/workflows/apps", method: "PUT"},
{handler: shuffle.GetWorkflows, path: "/api/v1/workflows", method: "GET"},
{handler: shuffle.SetNewWorkflow, path: "/api/v1/workflows", method: "POST"},
{handler: handleGetWorkflowqueue, path: "/api/v1/workflows/queue", method: "GET"},
{handler: handleGetWorkflowqueueConfirm, path: "/api/v1/workflows/queue/confirm", method: "POST"},
{handler: shuffle.HandleGetSchedules, path: "/api/v1/workflows/schedules", method: "GET"},
{handler: loadSpecificWorkflows, path: "/api/v1/workflows/download_remote", method: "POST"},
{handler: executeWorkflow, path: "/api/v1/workflows/123/execute", method: "GET"},
{handler: scheduleWorkflow, path: "/api/v1/workflows/123/schedule", method: "POST"},
{handler: stopSchedule, path: "/api/v1/workflows/123/schedule/abc", method: "DELETE"},
// createOutlookSub generates nil pointer exception
{handler: shuffle.HandleCreateOutlookSub, path: "/api/v1/workflows/123/outlook", method: "POST"},
// handleDeleteOutlookSub generates nil pointer exception
{handler: shuffle.HandleDeleteOutlookSub, path: "/api/v1/workflows/123/outlook/abc", method: "DELETE"},
{handler: shuffle.GetWorkflowExecutions, path: "/api/v1/workflows/123/executions", method: "GET"},
{handler: shuffle.AbortExecution, path: "/api/v1/workflows/123/executions/abc/abort", method: "GET"},
{handler: shuffle.GetSpecificWorkflow, path: "/api/v1/workflows/123", method: "GET"},
{handler: shuffle.SaveWorkflow, path: "/api/v1/workflows/123", method: "PUT"},
{handler: deleteWorkflow, path: "/api/v1/workflows/123", method: "DELETE"},
{handler: shuffle.HandleNewHook, path: "/api/v1/hooks/new", method: "POST"},
{handler: handleWebhookCallback, path: "/api/v1/hooks/123", method: "POST"},
{handler: shuffle.HandleDeleteHook, path: "/api/v1/hooks/123/delete", method: "DELETE"},
{handler: shuffle.HandleGetSpecificTrigger, path: "/api/v1/triggers/123", method: "GET"},
//{handler: shuffle.HandleGetSpecificStats, path: "/api/v1/stats/123", method: "GET"},
{handler: verifySwagger, path: "/api/v1/verify_swagger", method: "POST"},
{handler: verifySwagger, path: "/api/v1/verify_openapi", method: "POST"},
{handler: echoOpenapiData, path: "/api/v1/get_openapi_uri", method: "POST"},
{handler: echoOpenapiData, path: "/api/v1/validate_openapi", method: "POST"},
{handler: shuffle.ValidateSwagger, path: "/api/v1/validate_openapi", method: "POST"},
{handler: getOpenapi, path: "/api/v1/get_openapi", method: "GET"},
//{handler: shuffle.CleanupExecutions, path: "/api/v1/execution_cleanup", method: "GET"},
{handler: handleCloudSetup, path: "/api/v1/cloud/setup", method: "POST"},
{handler: shuffle.HandleGetOrgs, path: "/api/v1/orgs", method: "POST"},
{handler: shuffle.HandleGetFileContent, path: "/api/v1/files/{fileId}/content", method: "POST"},
}
//r := initHandlers(context.TODO())
initHandlers()
outerLoop:
for _, e := range handlers {
log.Printf("Endpoint: %#v", e.path)
req, err := http.NewRequest("OPTIONS", e.path, nil)
req.Header.Add("Origin", "http://localhost:3000")
req.Header.Add("Access-Control-Request-Method", "POST")
req.Header.Add("Access-Control-Request-Headers", "Content-Type, Accept, X-Requested-With, remember-me")
// OPTIONS /resource/foo
// Access-Control-Request-Method: DELETE
// Access-Control-Request-Headers: origin, x-requested-with
// Origin: https://foo.bar.org
if err != nil {
t.Errorf("Failure in OPTIONS setup: %s", err)
continue
}
rr := httptest.NewRecorder()
//timeoutHandler := http.TimeoutHandler(r, 2*time.Second, `Request Timeout`)
//timeoutHandler.ServeHTTP(rr, req)
funcName := getFunctionNameFromFunction(e.handler)
if status := rr.Code; status != http.StatusOK {
t.Errorf("%s handler returned wrong status code: got %v want %v",
funcName, status, http.StatusOK)
continue
}
want := map[string]string{
"Vary": "Origin",
"Access-Control-Allow-Headers": "Content-Type, Accept, X-Requested-With, Remember-Me",
"Access-Control-Allow-Methods": "POST",
"Access-Control-Allow-Credentials": "true",
"Access-Control-Allow-Origin": "http://localhost:3000",
}
// Remember to use canonical header name if accessing the headers array
// directly:
// v := r.Header[textproto.CanonicalMIMEHeaderKey("foo")]
// When using Header().Get(h), h will automatically be converted to canonical format.
for key, value := range want {
got := rr.Header().Get(key)
if got != value {
t.Errorf("%s handler returned wrong value for '%s' header: got '%v' want '%v'",
funcName, key, got, value)
continue outerLoop
}
}
}
}
func getFunctionNameFromFunction(f interface{}) string {
return runtime.FuncForPC(reflect.ValueOf(f).Pointer()).Name()
}
backend/go-app/run
Executable
+1
View File
@@ -0,0 +1 @@
go run main.go walkoff.go docker.go
backend/go-app/walkoff.go
+1059 -5179
View File
File diff suppressed because it is too large Load Diff
backend/tests/cache.sh
+15
View File
@@ -0,0 +1,15 @@
curl -XPOST http://192.168.3.8:5001/api/v1/orgs/6a6a99f5-6630-4f91-88ff-571c9f030ea0/set_cache -H "Authorization: Bearer e663cf93-7f10-4560-bef0-303f14aad982" -d '{
"workflow_id": "61825389-a125-43a5-9119-97c401e9934b",
"execution_id": "2c8ca1b7-6658-4742-86a1-105c9467702d",
"org_id": "6a6a99f5-6630-4f91-88ff-571c9f030ea0",
"key": "test",
"value": "THIS IS SOME DATA HELLO"
}'
curl -XPOST http://192.168.3.8:5001/api/v1/orgs/6a6a99f5-6630-4f91-88ff-571c9f030ea0/get_cache -H "Authorization: Bearer e663cf93-7f10-4560-bef0-303f14aad982" -d '{
"workflow_id": "61825389-a125-43a5-9119-97c401e9934b",
"execution_id": "2c8ca1b7-6658-4742-86a1-105c9467702d",
"org_id": "6a6a99f5-6630-4f91-88ff-571c9f030ea0",
"key": "test"
}'
backend/tests/dockerpull.sh
+5
View File
@@ -0,0 +1,5 @@
#!/bin/sh
#curl -XPOST http://localhost:5001/api/v1/get_docker_image -d '{"name":"frikky/shuffle:testing_1.0.0"}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" --output tarball.tgz
#curl -XPOST http://localhost:5001/api/v1/get_docker_image -d '{"name":"frikky/shuffle:testing_1.0.0"}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -O -J
#curl -XPOST http://localhost:5001/api/v1/get_docker_image -d '{"name":"frikky/shuffle:testing_1.0.0"}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" --output tarball.tgz
curl -XPOST http://localhost:5001/api/v1/get_docker_image -d '{"name":"frikky/shuffle:shuffle-tools_1.0.0"}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" --output tarball.tgz
backend/tests/files.sh
Executable
+22
View File
@@ -0,0 +1,22 @@
#curl http://192.168.3.6:5001/api/v1/files/create -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -d '{"filename": "file.txt", "org_id": "b199646b-16d2-456d-9fd6-b9972e929466", "workflow_id": "global"}'
#
#curl http://localhost:5001/api/v1/files/create -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -d '{"filename": "file.txt", "org_id": "b199646b-16d2-456d-9fd6-b9972e929466", "workflow_id": "global"}'
#
#echo
curl http://localhost:5001/api/v1/apps/upload -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -F 'shuffle_file=@files.sh'
#
#curl http://localhost:5001/api/v1/files/1915981b-b897-4db1-8a2e-44bc34cead3b/content -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
#curl http://localhost:5001/api/v1/files/e19cffe4-e2da-47e9-809e-904f5cb03687 -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
#curl -XDELETE http://localhost:5001/api/v1/files/e19cffe4-e2da-47e9-809e-904f5cb03687 -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
#r.HandleFunc("/api/v1/files/{fileId}/content", handleGetFileContent).Methods("GET", "OPTIONS")
#r.HandleFunc("/api/v1/files/create", handleCreateFile).Methods("POST", "OPTIONS")
#r.HandleFunc("/api/v1/files/{fileId}/upload", handleUploadFile).Methods("POST", "OPTIONS")
#r.HandleFunc("/api/v1/files/{fileId}", handleGetFileMeta).Methods("GET", "OPTIONS")
#r.HandleFunc("/api/v1/files/{fileId}", handleDeleteFile).Methods("DELETE", "OPTIONS")
#curl http://localhost:5001/api/v1/files/create -H "Authorization: Bearer c5b4c827-65ec-47f4-9e8a-234cdba38959" -d '{"filename": "rule2.yar", "org_id": "b4e88fe9-352b-47b4-b280-960181670acf", "workflow_id": "global", "namespace": "yara"}'
#curl http://localhost:5001/api/v1/files/5cb941ad-fa1c-4444-a685-92024b1fa31c/upload -H "Authorization: Bearer c5b4c827-65ec-47f4-9e8a-234cdba38959" -F 'shuffle_file=@upload.sh'
#curl http://localhost:5001/api/v1/files/namespaces/yara -H "Authorization: Bearer c5b4c827-65ec-47f4-9e8a-234cdba38959" --output rules.zip
backend/tests/hotload.sh
+1
View File
@@ -0,0 +1 @@
curl http://localhost:5001/api/v1/apps/run_hotload -H "Authorization: Bearer e08c6f22-9a55-4557-b008-04388cc51fb0"
backend/tests/list_files.sh
+1
View File
@@ -0,0 +1 @@
curl -XGET http://192.168.3.6:5001/api/v1/files -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
backend/tests/migrate_db.sh
+2
View File
@@ -0,0 +1,2 @@
curl -XPOST -v localhost:5001/api/v1/migrate_database -H 'Authorization: Bearer 0184d7be-33c1-4391-bf9c-dfb8508a4ea2'
backend/tests/schedules.sh
+1 -1
View File
@@ -1,2 +1,2 @@
# Fails cus of unmarshal
curl -XPOST http://localhost:5001/api/v1/workflows/1d9d8ce2-566e-4c3f-8a37-5d6c7d2000b5/schedule -d '{"name": "hey", "frequency": "*/1 * * * *", "execution_argument": "{\"test\": \"hey\"}"}' -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IjYwZjQwNjBlNThkNzVmZDNmNzBiZWZmODhjNzk0YTc3NTMyN2FhMzEiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJodHRwczovL3NodWZmbGVyLmlvL2FwaS92MS93b3JrZmxvd3MvMWQ5ZDhjZTItNTY2ZS00YzNmLThhMzctNWQ2YzdkMjAwMGI1L2V4ZWN1dGUiLCJhenAiOiIxMDMwNzY3ODIwNjE0MjQ2MTg0MjIiLCJlbWFpbCI6InNjaGVkdWxlckBzaHVmZmxlLTI0MTUxNy5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJleHAiOjE1NjU1Mjc1NTEsImlhdCI6MTU2NTUyMzk1MSwiaXNzIjoiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tIiwic3ViIjoiMTAzMDc2NzgyMDYxNDI0NjE4NDIyIn0.r0EDq9fjhf_5CPTiltyfk_L3uYJp577Uy0yYPcCAl2nv50_z_oUtbWGBpQLL8gcj-NGd3g4E52Qur8k6hCMIQweLS6WAb1279vGffEoCNDfkWb3Oy-yJGP1kzwLvqFJqnHLkSWYXNWvSyWnEimW8Rryx_m1BXS5wcA8l4NIr83kS7fPZrTwjnwFSeGSThwk91DVARzapQb8r0GEgOUyHZ1aBXnV98mikzSUt-5xFKe9eMdD22YJAj0Ru-DxAxs5nOqghX4PMRysWjshjOMrlR1piPWxqAmewp8YKZDCQ5gXskpeAFBDoULT971Wsx_NCohnJsFqx1JfPS9ZYMTW2oQ"
curl -XPOST http://localhost:5001/api/v1/workflows/1d9d8ce2-566e-4c3f-8a37-5d6c7d2000b5/schedule -d '{"name": "hey", "frequency": "*/1 * * * *", "execution_argument": "{\"test\": \"hey\"}"}' -H "Authorization: Bearer WUT"
backend/tests/stop_execution.sh
+1
View File
@@ -0,0 +1 @@
curl "http://localhost:5001/api/v1/environments/Shuffle/stop" -H "Authorization: Bearer e663cf93-7f10-4560-bef0-303f14aad982"
backend/tests/test_wrappers.py
+210
View File
@@ -0,0 +1,210 @@
import re
import json
def parse_nested_param(string, level):
"""
Generate strings contained in nested (), indexing i = level
"""
if len(re.findall("\(", string)) == len(re.findall("\)", string)):
LeftRightIndex = [x for x in zip(
[Left.start()+1 for Left in re.finditer('\(', string)],
reversed([Right.start() for Right in re.finditer('\)', string)]))]
elif len(re.findall("\(", string)) > len(re.findall("\)", string)):
return parse_nested_param(string + ')', level)
elif len(re.findall("\(", string)) < len(re.findall("\)", string)):
return parse_nested_param('(' + string, level)
else:
return 'Failed to parse params'
try:
return [string[LeftRightIndex[level][0]:LeftRightIndex[level][1]]]
except IndexError:
return [string[LeftRightIndex[level+1][0]:LeftRightIndex[level+1][1]]]
# Parses the deepest part
def maxDepth(S):
current_max = 0
max = 0
n = len(S)
# Traverse the input string
for i in range(n):
if S[i] == '(':
current_max += 1
if current_max > max:
max = current_max
elif S[i] == ')':
if current_max > 0:
current_max -= 1
else:
return -1
# finally check for unbalanced string
if current_max != 0:
return -1
return max-1
def parse_type(data, thistype):
if data == None:
return "Empty"
if "int" in thistype:
try:
return int(data)
except ValueError:
print("ValueError while casting %s" % data)
return data
if "lower" in thistype:
return data.lower()
if "upper" in thistype:
return data.upper()
if "trim" in thistype:
return data.strip()
if "strip" in thistype:
return data.strip()
if "split" in thistype:
# Should be able to split anything
return data.split()
if "len" in thistype or "length" in thistype:
return len(data)
if "parse" in thistype:
splitvalues = []
default_error = """Error. Expected syntax: parse(["hello","test1"],0:1)"""
if "," in data:
splitvalues = data.split(",")
for item in range(len(splitvalues)):
splitvalues[item] = splitvalues[item].strip()
else:
return default_error
lastsplit = []
if ":" in splitvalues[-1]:
lastsplit = splitvalues[-1].split(":")
else:
try:
lastsplit = [int(splitvalues[-1])]
except ValueError:
return default_error
try:
parsedlist = ",".join(splitvalues[0:-1])
print(parsedlist)
print(lastsplit)
if len(lastsplit) > 1:
tmp = json.loads(parsedlist)[int(lastsplit[0]):int(lastsplit[1])]
else:
tmp = json.loads(parsedlist)[lastsplit[0]]
print(tmp)
return tmp
except IndexError as e:
return default_error
# Parses the INNER value and recurses until everything is done
def parse_wrapper(data):
try:
if "(" not in data or ")" not in data:
return data
except TypeError:
return data
print("Running %s" % data)
# Look for the INNER wrapper first, then move out
wrappers = ["int", "number", "lower", "upper", "trim", "strip", "split", "parse", "len", "length"]
found = False
for wrapper in wrappers:
if wrapper not in data.lower():
continue
found = True
break
if not found:
return data
# Do stuff here.
innervalue = parse_nested_param(data, maxDepth(data)-0)
outervalue = parse_nested_param(data, maxDepth(data)-1)
print("INNER: ", outervalue)
print("OUTER: ", outervalue)
if outervalue != innervalue:
#print("Outer: ", outervalue, " inner: ", innervalue)
for key in range(len(innervalue)):
# Replace OUTERVALUE[key] with INNERVALUE[key] in data.
print("Replace %s with %s in %s" % (outervalue[key], innervalue[key], data))
data = data.replace(outervalue[key], innervalue[key])
else:
for thistype in wrappers:
if thistype not in data.lower():
continue
parsed_value = parse_type(innervalue[0], thistype)
return parsed_value
print("DATA: %s\n" % data)
return parse_wrapper(data)
def parse_wrapper_start(data):
newdata = []
newstring = ""
record = True
paranCnt = 0
for char in data:
if char == "(":
paranCnt += 1
if not record:
record = True
if record:
newstring += char
if paranCnt == 0 and char == " ":
newdata.append(newstring)
newstring = ""
record = True
if char == ")":
paranCnt -= 1
if paranCnt == 0:
record = False
if len(newstring) > 0:
newdata.append(newstring)
parsedlist = []
non_string = False
for item in newdata:
ret = parse_wrapper(item)
if not isinstance(ret, str):
non_string = True
parsedlist.append(ret)
if len(parsedlist) > 0 and not non_string:
return " ".join(parsedlist)
elif len(parsedlist) == 1 and non_string:
return parsedlist[0]
else:
print("Casting back to string because multi: ", parsedlist)
newlist = []
for item in parsedlist:
try:
newlist.append(str(item))
except ValueError:
newlist.append("parsing_error")
return " ".join(newlist)
data = "split(hello there)"
data = """parse(["testing", "what", "is this"], 0:2)"""
#data = "int(int(2))"
print("RET: ", parse_wrapper_start(data))
backend/tests/upload.sh
+1
View File
@@ -0,0 +1 @@
#
backend/tests/users.sh
+13
View File
@@ -0,0 +1,13 @@
curl http://localhost:5001/api/v1/users/register -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" -d '{"username": "username1", "password": ""}'
echo
curl http://localhost:5001/api/v1/users -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e"
echo UPDATE
curl -XPUT http://localhost:5001/api/v1/users/updateuser -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" -d '{"user_id": "id", "role": "admin"}'
echo
curl -XDELETE http://localhost:5001/api/v1/users/userid -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e"
echo
curl -XPOST http://localhost:5001/api/v1/users/generateapikey -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" -d '{"user_id": "390efa79-73a3-454b-8b1d-38f56eec14ad"}'
backend/tests/validate_app_values.sh
+14
View File
@@ -0,0 +1,14 @@
# ExecutionOrg MUST be executing.
curl -XPOST http://localhost:5001/api/v1/orgs/b199646b-16d2-456d-9fd6-b9972e929466/validate_app_values -d '{
"append": true,
"workflow_check": true,
"authorization": "1aae630c-ccaf-4cb5-87f9-8a9e0a9afd11",
"execution_ref": "c59ff288-4f02-4d02-b839-133d55c7fdf0",
"org_id": "b199646b-16d2-456d-9fd6-b9972e929466",
"values": [{
"app": "testing",
"action": "repeat_back_to_me",
"parameternames": ["call"],
"parametervalues": ["hey", "ho", "lets", "go"]
}]
}'
docker-compose.yml
+73 -36
View File
@@ -2,7 +2,7 @@ version: '3'
services:
frontend:
#build: ./frontend
image: ghcr.io/frikky/shuffle-frontend:0.8.56
image: ghcr.io/frikky/shuffle-frontend:nightly
container_name: shuffle-frontend
hostname: shuffle-frontend
ports:
@@ -16,8 +16,8 @@ services:
depends_on:
- backend
backend:
#build: ./backend
image: ghcr.io/frikky/shuffle-backend:0.8.56
build: ./backend
image: ghcr.io/frikky/shuffle-backend:nightly
container_name: shuffle-backend
hostname: ${BACKEND_HOSTNAME}
# Here for debugging:
@@ -25,27 +25,21 @@ services:
- "${BACKEND_PORT}:5001"
networks:
- shuffle
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ${SHUFFLE_APP_HOTLOAD_LOCATION}:/shuffle-apps
- ${SHUFFLE_FILE_LOCATION}:/shuffle-files
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ${SHUFFLE_APP_HOTLOAD_LOCATION}:/shuffle-apps:z
- ${SHUFFLE_FILE_LOCATION}:/shuffle-files:z
#- ${SHUFFLE_OPENSEARCH_CERTIFICATE_FILE}:/shuffle-files/es_certificate
env_file: .env
environment:
- DATASTORE_EMULATOR_HOST=shuffle-database:8000
- SHUFFLE_APP_HOTLOAD_FOLDER=/shuffle-apps
- SHUFFLE_FILE_LOCATION=/shuffle-files
- ORG_ID=${ORG_ID}
- SHUFFLE_APP_DOWNLOAD_LOCATION=${SHUFFLE_APP_DOWNLOAD_LOCATION}
- SHUFFLE_DEFAULT_USERNAME=${SHUFFLE_DEFAULT_USERNAME}
- SHUFFLE_DEFAULT_PASSWORD=${SHUFFLE_DEFAULT_PASSWORD}
- SHUFFLE_DEFAULT_APIKEY=${SHUFFLE_DEFAULT_APIKEY}
- HTTP_PROXY=${SHUFFLE_HTTP_PROXY}
- HTTPS_PROXY=${SHUFFLE_HTTPS_PROXY}
restart: unless-stopped
depends_on:
- database
#depends_on:
#- opensearch #Not necessary because dependancy is handled within the backend itself instead
#- database
orborus:
#build: ./functions/onprem/orborus
image: ghcr.io/frikky/shuffle-orborus:0.8.5
image: ghcr.io/frikky/shuffle-orborus:nightly
container_name: shuffle-orborus
hostname: shuffle-orborus
networks:
@@ -53,33 +47,76 @@ services:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- SHUFFLE_APP_SDK_VERSION=0.8.51
- SHUFFLE_WORKER_VERSION=0.8.54
- ORG_ID=${ORG_ID}
- SHUFFLE_WORKER_VERSION=nightly
- ENVIRONMENT_NAME=${ENVIRONMENT_NAME}
- BASE_URL=http://${OUTER_HOSTNAME}:${BACKEND_PORT}
- BASE_URL=http://${OUTER_HOSTNAME}:5001
- DOCKER_API_VERSION=1.40
- HTTP_PROXY=${SHUFFLE_HTTP_PROXY}
- HTTPS_PROXY=${SHUFFLE_HTTPS_PROXY}
- SHUFFLE_PASS_WORKER_PROXY=${SHUFFLE_PASS_WORKER_PROXY}
- SHUFFLE_ORBORUS_EXECUTION_TIMEOUT=600
- SHUFFLE_BASE_IMAGE_NAME=${SHUFFLE_BASE_IMAGE_NAME}
- SHUFFLE_BASE_IMAGE_REGISTRY=${SHUFFLE_BASE_IMAGE_REGISTRY}
- SHUFFLE_BASE_IMAGE_TAG_SUFFIX=${SHUFFLE_BASE_IMAGE_TAG_SUFFIX}
- CLEANUP=${SHUFFLE_CONTAINER_AUTO_CLEANUP}
- HTTP_PROXY=${HTTP_PROXY}
- HTTPS_PROXY=${HTTPS_PROXY}
- SHUFFLE_PASS_WORKER_PROXY=${SHUFFLE_PASS_WORKER_PROXY}
- SHUFFLE_PASS_APP_PROXY=${SHUFFLE_PASS_APP_PROXY}
- SHUFFLE_SCALE_REPLICAS=1
- SHUFFLE_SWARM_CONFIG=runn
- SHUFFLE_SWARM_NETWORK_NAME=shuffle_swarm_executions
#- DOCKER_HOST=tcp://docker-socket-proxy:2375
restart: unless-stopped
database:
#build: ./backend/database
image: frikky/shuffle:database
container_name: shuffle-database
hostname: shuffle-database
security_opt:
- seccomp:unconfined
opensearch:
image: opensearchproject/opensearch:1.2.4
hostname: shuffle-opensearch
container_name: shuffle-opensearch
environment:
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
- cluster.routing.allocation.disk.threshold_enabled=false
- cluster.name=shuffle-cluster
- node.name=shuffle-opensearch
- discovery.seed_hosts=shuffle-opensearch
- cluster.initial_master_nodes=shuffle-opensearch
- node.store.allow_mmap=false
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- ${DB_LOCATION}:/usr/share/opensearch/data:z
ports:
- "8000:8000"
- 9200:9200
networks:
- shuffle
restart: unless-stopped
volumes:
- ${DB_LOCATION}:/etc/shuffle
#docker-socket-proxy:
# image: tecnativa/docker-socket-proxy
# privileged: true
# environment:
# - SERVICES=1
# - TASKS=1
# - NETWORKS=1
# - NODES=1
# - BUILD=1
# - IMAGES=1
# - GRPC=1
# - CONTAINERS=1
# - PLUGINS=1
# - SYSTEM=1
# - VOLUMES=1
# - INFO=1
# - DISTRIBUTION=1
# - POST=1
# - AUTH=1
# - SECRETS=1
# volumes:
# - /var/run/docker.sock:/var/run/docker.sock
# networks:
# - shuffle
networks:
shuffle:
driver: bridge
#driver: overlay
frontend/Dockerfile
+3 -6
View File
@@ -1,5 +1,5 @@
# Build environment
FROM node as builder
FROM node:14 as builder
RUN mkdir /usr/src/app
@@ -8,7 +8,7 @@ ENV PATH /usr/src/app/node_modules/.bin:$PATH
COPY package.json /usr/src/app/package.json
#RUN npm install --verbose
RUN yarn config set "strict-ssl" false -g
RUN yarn install
# copy only required files to not trigger rebuilding every time
@@ -18,14 +18,11 @@ COPY ./src /usr/src/app/src/
COPY ./*.sh /usr/src/app/
COPY ./*.json /usr/src/app/
# There were issues with the webpack installer from package.json
RUN rm -rf /usr/src/app/node_modules/webpack
#RUN yarn add webpack@4.42.0
RUN yarn build
# Production environment
FROM nginx:latest
FROM nginx:1.21.5
RUN mkdir -p /usr/share/nginx/html/build
RUN mkdir -p /usr/share/nginx/html/css
frontend/README.md
+1
View File
@@ -1,4 +1,5 @@
# Certificate:
Creating a localhost certificate:
```
frontend/confd/templates/nginx.conf
+22
View File
@@ -28,6 +28,28 @@ http {
server {
listen 80;
server_name "localhost";
#location /static/js/* {
# # avoid clickjacking
# add_header X-Frame-Options DENY;
# add_header X-Content-Type-Options nosniff;
# add_header ;
# # block MIME sniffing
# # security headers
# add_header X-XSS-Protection "1; mode=block";
# # add_header Content-Security-Policy "default-src 'self'";
# add_header Referrer-Policy "no-referrer";
# server_tokens off;
# root /usr/share/nginx/html;
# gzip_static on;
# expires 1y;
# add_header Cache-Control public;
# add_header ETag "";
# try_files $uri /index.html;
#}
location / {
# avoid clickjacking
add_header X-Frame-Options DENY;
frontend/package-lock.json
Generated
-20920
View File
File diff suppressed because it is too large Load Diff
frontend/package.json
+39 -17
View File
@@ -1,42 +1,54 @@
{
"name": "shuffler",
"homepage": "https://shuffler.io",
"version": "0.8.53",
"version": "0.9.61",
"private": true,
"dependencies": {
"@babel/core": "^7.15.8",
"@emotion/is-prop-valid": "^1.1.1",
"@emotion/react": "^11.7.0",
"@emotion/styled": "^11.6.0",
"@material-ui/core": "^4.5.2",
"@material-ui/data-grid": "^4.0.0-alpha.22",
"@material-ui/icons": "^4.5.1",
"@material-ui/lab": "^4.0.0-alpha.58",
"@material-ui/styles": "^4.5.2",
"@material-ui/utils": "^4.11.2",
"@metamask/detect-provider": "^1.2.0",
"@mui/icons-material": "^5.2.1",
"@mui/material": "^5.2.3",
"@uiw/react-codemirror": "^3.2.1",
"@use-it/interval": "^1.0.0",
"babel-eslint": "^10.1.0",
"class-transformer": "^0.3.1",
"create-react-app": "^2.0.3",
"cytoscape": "^3.11.0",
"class-transformer": "^0.4.0",
"create-react-app": "^4.0.3",
"cytoscape": "^3.15.1",
"cytoscape-clipboard": "^2.2.1",
"cytoscape-cxtmenu": "^3.1.1",
"cytoscape-edgehandles": "^3.6.0",
"cytoscape-grid-guide": "~2.1.2",
"cytoscape-grid-guide": "~2.3.3",
"cytoscape-node-html-label": "^1.1.5",
"cytoscape-panzoom": "^2.5.2",
"cytoscape-undo-redo": "^1.3.2",
"d3": "~4.10.0",
"d3": "^7.1.1",
"dotenv": "^6.1.0",
"downshift": "^3.3.5",
"github-markdown-css": "^3.0.1",
"import": "0.0.6",
"interweave": "^11.2.0",
"material-icons": "^0.3.1",
"material-icons": "^0.7.7",
"material-icons-react": "^1.0.4",
"material-ui-chip-input": "^2.0.0-beta.2",
"material-ui-nested-menu-item": "^1.0.2",
"md5-file": "^4.0.0",
"mdbreact": "^4.21.1",
"moment": "~2.20.1",
"moment": "^2.29.1",
"react": "^16.14.0",
"react-alert": "^5.5.0",
"react-alert-template-basic": "^1.0.0",
"react-avatar-editor": "^11.1.0",
"react-beforeunload": "^2.2.1",
"react-chartjs-2": "^2.8.0",
"react-chartjs-2": "^2.11.1",
"react-cookie": "^4.0.1",
"react-cytoscapejs": "^1.2.0",
"react-device-detect": "^1.9.10",
@@ -50,27 +62,36 @@
"react-markdown": "^4.2.2",
"react-markdown-github": "^3.3.1",
"react-powerhooks": "0.0.7",
"react-router": "^4.3.1",
"react-router-dom": "^4.3.1",
"react-scripts": "^3.4.1",
"react-router": "6.2.1",
"react-router-dom": "6.2.1",
"react-scripts": "^4.0.1",
"react-shepherd": "^3.3.6",
"reactstrap": "^7.1.0",
"reaviz": "^12.1.0",
"shellwords": "^0.1.1",
"simplebar": "^4.2.3",
"styled-components": "^4.4.0",
"webpack": "^4.42.0",
"webpack": "^4.44.2",
"websocket": "^1.0.30",
"yaml": "^1.7.2",
"yamljs": "^0.3.0",
"zone.js": "~0.8.26"
"zone.js": "~0.11.4"
},
"scripts": {
"start": "set HTTPS=true&&react-scripts start",
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"eject": "react-scripts eject"
"eject": "react-scripts eject",
"lint": "eslint 'src/**/*.{tsx,ts,js,jsx}'",
"lint_file": "eslint 'src/views/AngularWorkflow.jsx'"
},
"eslintConfig": {
"extends": "react-app"
"extends": "react-app",
"rules": {
"jsx-a11y/img-redundant-alt": "off",
"no-redeclare": "off",
"no-loop-func": "off"
}
},
"browserslist": [
">0.2%",
@@ -79,6 +100,7 @@
"not op_mini all"
],
"devDependencies": {
"prettier": "2.4.1",
"promise-window": "^1.2.1"
}
}
frontend/public/images/Shuffle_logo.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 217 KiB

frontend/public/images/Shuffle_logo_new.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

frontend/public/images/btn_google_light_focus_ios.svg
+44
View File
@@ -0,0 +1,44 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg width="46px" height="46px" viewBox="0 0 46 46" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:sketch="http://www.bohemiancoding.com/sketch/ns">
<!-- Generator: Sketch 3.3.3 (12081) - http://www.bohemiancoding.com/sketch -->
<title>btn_google_light_focus_ios</title>
<desc>Created with Sketch.</desc>
<defs>
<filter x="-50%" y="-50%" width="200%" height="200%" filterUnits="objectBoundingBox" id="filter-1">
<feOffset dx="0" dy="1" in="SourceAlpha" result="shadowOffsetOuter1"></feOffset>
<feGaussianBlur stdDeviation="0.5" in="shadowOffsetOuter1" result="shadowBlurOuter1"></feGaussianBlur>
<feColorMatrix values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.168 0" in="shadowBlurOuter1" type="matrix" result="shadowMatrixOuter1"></feColorMatrix>
<feOffset dx="0" dy="0" in="SourceAlpha" result="shadowOffsetOuter2"></feOffset>
<feGaussianBlur stdDeviation="0.5" in="shadowOffsetOuter2" result="shadowBlurOuter2"></feGaussianBlur>
<feColorMatrix values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.084 0" in="shadowBlurOuter2" type="matrix" result="shadowMatrixOuter2"></feColorMatrix>
<feMerge>
<feMergeNode in="shadowMatrixOuter1"></feMergeNode>
<feMergeNode in="shadowMatrixOuter2"></feMergeNode>
<feMergeNode in="SourceGraphic"></feMergeNode>
</feMerge>
</filter>
<rect id="path-2" x="0" y="0" width="40" height="40" rx="2"></rect>
</defs>
<g id="Google-Button" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd" sketch:type="MSPage">
<g id="9-PATCH" sketch:type="MSArtboardGroup" transform="translate(-668.000000, -160.000000)"></g>
<g id="btn_google_light_focus" sketch:type="MSArtboardGroup" transform="translate(-1.000000, -1.000000)">
<rect id="Rectangle-14" fill-opacity="0.3" fill="#4285F4" sketch:type="MSShapeGroup" x="1" y="1" width="46" height="46"></rect>
<g id="button" sketch:type="MSLayerGroup" transform="translate(4.000000, 4.000000)" filter="url(#filter-1)">
<g id="button-bg">
<use fill="#FFFFFF" fill-rule="evenodd" sketch:type="MSShapeGroup" xlink:href="#path-2"></use>
<use fill="none" xlink:href="#path-2"></use>
<use fill="none" xlink:href="#path-2"></use>
<use fill="none" xlink:href="#path-2"></use>
</g>
</g>
<g id="logo_googleg_48dp" sketch:type="MSLayerGroup" transform="translate(15.000000, 15.000000)">
<path d="M17.64,9.20454545 C17.64,8.56636364 17.5827273,7.95272727 17.4763636,7.36363636 L9,7.36363636 L9,10.845 L13.8436364,10.845 C13.635,11.97 13.0009091,12.9231818 12.0477273,13.5613636 L12.0477273,15.8195455 L14.9563636,15.8195455 C16.6581818,14.2527273 17.64,11.9454545 17.64,9.20454545 L17.64,9.20454545 Z" id="Shape" fill="#4285F4" sketch:type="MSShapeGroup"></path>
<path d="M9,18 C11.43,18 13.4672727,17.1940909 14.9563636,15.8195455 L12.0477273,13.5613636 C11.2418182,14.1013636 10.2109091,14.4204545 9,14.4204545 C6.65590909,14.4204545 4.67181818,12.8372727 3.96409091,10.71 L0.957272727,10.71 L0.957272727,13.0418182 C2.43818182,15.9831818 5.48181818,18 9,18 L9,18 Z" id="Shape" fill="#34A853" sketch:type="MSShapeGroup"></path>
<path d="M3.96409091,10.71 C3.78409091,10.17 3.68181818,9.59318182 3.68181818,9 C3.68181818,8.40681818 3.78409091,7.83 3.96409091,7.29 L3.96409091,4.95818182 L0.957272727,4.95818182 C0.347727273,6.17318182 0,7.54772727 0,9 C0,10.4522727 0.347727273,11.8268182 0.957272727,13.0418182 L3.96409091,10.71 L3.96409091,10.71 Z" id="Shape" fill="#FBBC05" sketch:type="MSShapeGroup"></path>
<path d="M9,3.57954545 C10.3213636,3.57954545 11.5077273,4.03363636 12.4404545,4.92545455 L15.0218182,2.34409091 C13.4631818,0.891818182 11.4259091,0 9,0 C5.48181818,0 2.43818182,2.01681818 0.957272727,4.95818182 L3.96409091,7.29 C4.67181818,5.16272727 6.65590909,3.57954545 9,3.57954545 L9,3.57954545 Z" id="Shape" fill="#EA4335" sketch:type="MSShapeGroup"></path>
<path d="M0,0 L18,0 L18,18 L0,18 L0,0 Z" id="Shape" sketch:type="MSShapeGroup"></path>
</g>
<g id="handles_square" sketch:type="MSLayerGroup"></g>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.4 KiB

frontend/public/images/detectionframework.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 109 KiB

frontend/public/index.html
+8 -6
View File
@@ -1,13 +1,15 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<link rel="shortcut icon" href="%PUBLIC_URL%/favicon.ico">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="theme-color" content="#000000">
<link rel="manifest" href="%PUBLIC_URL%/manifest.json">
<meta charset="utf-8" />
<link rel="shortcut icon" href="%PUBLIC_URL%/favicon.ico" />
<meta
name="viewport"
content="width=device-width, initial-scale=1, shrink-to-fit=no"
/>
<meta name="theme-color" content="#000000" />
<link rel="manifest" href="%PUBLIC_URL%/manifest.json" />
<title>Shuffle</title>
</head>
<body>
<div id="root"></div>
frontend/run.sh
+2 -2
View File
@@ -1,11 +1,11 @@
#!/bin/sh
docker stop shuffle-frontend
docker rm shuffle-frontend
docker rmi frikky/shuffle:frontend
#docker rmi ghcr.io/frikky/shuffle-frontend:nightly
echo "Running build for website"
#sudo npm run build
docker build . -t frikky/shuffle:frontend
docker build . -t ghcr.io/frikky/shuffle-frontend:nightly
echo "Starting server"
# Rerun build locally for it to update :)
frontend/src/App.jsx
+650 -133
View File
@@ -1,170 +1,687 @@
import React, { useState, useEffect } from 'react';
import React, { useState, useEffect } from "react";
import { Route } from 'react-router';
import { BrowserRouter } from 'react-router-dom';
import { CookiesProvider } from 'react-cookie';
import { removeCookies, useCookies } from 'react-cookie';
//import { Route, Routes } from "react-router";
import { Route, Routes, BrowserRouter } from "react-router-dom";
import { CookiesProvider } from "react-cookie";
import { removeCookies, useCookies } from "react-cookie";
import EditSchedule from "./views/EditSchedule";
import Schedules from "./views/Schedules";
import Webhooks from "./views/Webhooks";
import Workflows from "./views/Workflows";
import GettingStarted from "./views/GettingStarted";
import EditWebhook from "./views/EditWebhook";
import AngularWorkflow from "./views/AngularWorkflow";
import Header from './components/Header';
import Apps from './views/Apps';
import AppCreator from './views/AppCreator';
import Contact from './views/Contact';
import Oauth2 from './views/Oauth2';
import Header from "./components/Header";
import theme from "./theme";
import Apps from "./views/Apps";
import AppCreator from "./views/AppCreator";
import Dashboard from "./views/Dashboard";
import Dashboard from "./views/Dashboard.jsx";
import AdminSetup from "./views/AdminSetup";
import Admin from "./views/Admin";
import Docs from "./views/Docs";
import Introduction from "./views/Introduction";
import SetAuthentication from "./views/SetAuthentication";
import SetAuthenticationSSO from "./views/SetAuthenticationSSO";
import LandingPageNew from "./views/LandingpageNew";
import LoginPage from "./views/LoginPage";
import SettingsPage from "./views/SettingsPage";
import KeepAlive from "./views/KeepAlive.jsx";
import { createMuiTheme, MuiThemeProvider } from '@material-ui/core/styles';
import MyView from "./views/MyView";
import { createMuiTheme, MuiThemeProvider } from "@material-ui/core/styles";
import FrameworkWrapper from "./views/FrameworkWrapper.jsx";
import ScrollToTop from "./components/ScrollToTop";
import AlertTemplate from "./components/AlertTemplate";
import { positions, Provider } from "react-alert";
import { useAlert, positions, Provider } from "react-alert";
import { isMobile } from "react-device-detect";
import detectEthereumProvider from "@metamask/detect-provider";
import Drift from "react-driftjs";
// Production - backend proxy forwarding in nginx
var globalUrl = window.location.origin
var globalUrl = window.location.origin;
// CORS used for testing purposes. Should only happen with specific port and http
if (window.location.protocol == "http:" && window.location.port === "3000") {
globalUrl = "http://localhost:5001"
//globalUrl = "http://localhost:5002"
if (window.location.port === "3000") {
globalUrl = "http://localhost:5001";
//globalUrl = "http://localhost:5002"
}
const theme = createMuiTheme({
palette: {
primary: {
main: "#f85a3e"
},
secondary: {
main: '#e8eaf6',
},
surfaceColor: "#27292d",
inputColor: "#383B40"
},
typography: {
useNextVariants: true
},
overrides: {
MuiMenu: {
list: {
backgroundColor: "#383B40",
},
},
},
});
if (globalUrl.includes("githubpreview.dev")) {
//globalUrl = globalUrl.replace("3000", "5001")
globalUrl = "https://frikky-shuffle-5gvr4xx62w64-5001.githubpreview.dev"
}
console.log("global: ", globalUrl)
// FIXME - set client side cookies
const App = (message, props) => {
const [userdata, setUserData] = useState({});
//const [homePage, ] = useState(true);
const [cookies, setCookie, removeCookie] = useCookies([]);
const [isLoggedIn, setIsLoggedIn] = useState(false);
const [dataset, setDataset] = useState(false);
const [isLoaded, setIsLoaded] = useState(false);
useEffect(() => {
if (dataset === false) {
checkLogin()
setDataset(true)
}
})
const [userdata, setUserData] = useState({});
const [notifications, setNotifications] = useState([])
const [cookies, setCookie, removeCookie] = useCookies([])
const [isLoggedIn, setIsLoggedIn] = useState(false)
const [dataset, setDataset] = useState(false)
const [isLoaded, setIsLoaded] = useState(false)
const [curpath, setCurpath] = useState(
typeof window === "undefined" || window.location === undefined
? ""
: window.location.pathname
)
if (isLoaded && !isLoggedIn && (!window.location.pathname.startsWith("/login") && (!window.location.pathname.startsWith("/docs") && (!window.location.pathname.startsWith("/adminsetup"))))) {
window.location = "/login"
useEffect(() => {
if (dataset === false) {
getUserNotifications();
checkLogin();
setDataset(true);
}
}, []);
if (
isLoaded &&
!isLoggedIn &&
!window.location.pathname.startsWith("/login") &&
!window.location.pathname.startsWith("/docs") &&
!window.location.pathname.startsWith("/support") &&
!window.location.pathname.startsWith("/detectionframework") &&
!window.location.pathname.startsWith("/appframework") &&
!window.location.pathname.startsWith("/adminsetup") &&
!window.location.pathname.startsWith("/usecases")
) {
window.location = "/login";
}
const getUserNotifications = () => {
fetch(`${globalUrl}/api/v1/users/notifications`, {
credentials: "include",
headers: {
"Content-Type": "application/json",
},
cors: "cors",
})
.then((response) => response.json())
.then((responseJson) => {
if (
responseJson.success === true &&
responseJson.notifications !== null &&
responseJson.notifications !== undefined &&
responseJson.notifications.length > 0
) {
//console.log("RESP: ", responseJson)
setNotifications(responseJson.notifications);
}
})
.catch((error) => {
console.log("Failed getting notifications for user: ", error);
});
};
const checkLogin = () => {
var baseurl = globalUrl;
fetch(`${globalUrl}/api/v1/getinfo`, {
credentials: "include",
headers: {
"Content-Type": "application/json",
},
})
.then((response) => response.json())
.then((responseJson) => {
var userInfo = {};
if (responseJson.success === true) {
console.log(responseJson);
userInfo = responseJson;
setIsLoggedIn(true);
//console.log("Cookies: ", cookies)
// Updating cookie every request
for (var key in responseJson["cookies"]) {
setCookie(
responseJson["cookies"][key].key,
responseJson["cookies"][key].value,
{ path: "/" }
);
}
}
// Handling Ethereum update
{/*
detectEthereumProvider().then((provider) => {
if (
provider &&
userInfo.eth_info !== undefined &&
userInfo.eth_info !== null
) {
if (
userInfo.eth_info.account !== undefined &&
userInfo.eth_info.account !== null &&
userInfo.eth_info.account.length === 0
) {
userInfo.eth_info = {};
var method = "eth_requestAccounts";
var params = [];
provider
.request({
method: method,
params,
})
.then((result) => {
if (
result !== undefined &&
result !== null &&
result.length > 0
) {
userInfo.eth_info.account = result[0];
// Getting and setting balance for the current user
method = "eth_getBalance";
params = [userInfo.eth_info.account, "latest"];
provider
.request({
method: method,
params,
})
.then((result) => {
if (
result !== undefined &&
result !== null &&
result.length > 0
) {
userInfo.parsed_balance =
result / 1000000000000000000;
} else {
alert.error("Couldn't find balance: ", result);
}
// The result varies by RPC method.
// For example, this method will return a transaction hash hexadecimal string on success.
})
.catch((error) => {
// If the request fails, the Promise will reject with an error.
alert.error(
"Failed getting info from ethereum API: " + error
);
});
} else {
alert.error("Couldn't find any user: ", result);
}
})
.catch((error) => {
// If the request fails, the Promise will reject with an error.
alert.error(
"Failed getting info from ethereum API: " + error
);
});
}
// Register hooks here
provider.on("message", (event) => {
alert.info("Message from MetaMask: ", event);
});
provider.on("chainChanged", (chainId) => {
console.log("Changed chain to: ", chainId);
method = "eth_getBalance";
params = [userInfo.eth_info.account, "latest"];
provider
.request({
method: method,
params,
})
.then((result) => {
console.log("Got result: ", result);
if (result !== undefined && result !== null) {
userInfo.eth_info.balance = result;
userInfo.eth_info.parsed_balance =
result / 1000000000000000000;
console.log("INFO: ", userInfo);
setUserData(userInfo);
} else {
alert.error("Couldn't find balance: ", result);
}
})
.catch((error) => {
// If the request fails, the Promise will reject with an error.
alert.error(
"Failed getting info from ethereum API: " + error
);
});
});
}
});
if (
userInfo.eth_info !== undefined &&
userInfo.eth_info.balance !== undefined
) {
//console.log(userInfo.eth_info.balance)
userInfo.eth_info.parsed_balance =
userInfo.eth_info.balance / 1000000000000000000;
}
*/}
//console.log("USER: ", userInfo)
setUserData(userInfo);
setIsLoaded(true);
})
.catch((error) => {
setIsLoaded(true);
});
};
// Dumb for content load (per now), but good for making the site not suddenly reload parts (ajax thingies)
const options = {
timeout: 9000,
position: positions.BOTTOM_LEFT,
};
const handleFirstInteraction = (event) => {
console.log("First interaction: ", event)
}
const checkLogin = () => {
var baseurl = globalUrl
fetch(baseurl + "/api/v1/users/getinfo", {
credentials: "include",
headers: {
'Content-Type': 'application/json',
},
})
.then(response => response.json())
.then(responseJson => {
if (responseJson.success === true) {
//console.log(responseJson.success)
setUserData(responseJson)
setIsLoggedIn(true)
//console.log("Cookies: ", cookies)
// Updating cookie every request
for (var key in responseJson["cookies"]) {
setCookie(responseJson["cookies"][key].key, responseJson["cookies"][key].value, { path: "/" })
const includedData =
window.location.pathname === "/home" ||
window.location.pathname === "/features" ? (
<div>
<Routes>
<Route
exact
path="/home"
render={(props) => <LandingPageNew isLoaded={isLoaded} {...props} />}
/>
</Routes>
</div>
) : (
<div
style={{
backgroundColor: "#1F2023",
color: "rgba(255, 255, 255, 0.65)",
minHeight: "100vh",
}}
>
<ScrollToTop
getUserNotifications={getUserNotifications}
setCurpath={setCurpath}
/>
{!isLoaded ? null :
userdata.chat_disabled === true ? null :
<Drift
appId="zfk9i7w3yizf"
attributes={{
name: userdata.username === undefined || userdata.username === null ? "OSS user" : `${userdata.username} - OSS`,
}}
eventHandlers={[
{
event: "conversation:firstInteraction",
function: handleFirstInteraction
},
]}
/>
}
}
setIsLoaded(true)
})
.catch(error => {
setIsLoaded(true)
});
}
<Header
notifications={notifications}
setNotifications={setNotifications}
checkLogin={checkLogin}
cookies={cookies}
removeCookie={removeCookie}
isLoaded={isLoaded}
globalUrl={globalUrl}
setIsLoggedIn={setIsLoggedIn}
isLoggedIn={isLoggedIn}
userdata={userdata}
{...props}
/>
<div style={{ height: 60 }} />
<Routes>
<Route
exact
path="/login"
element={
<LoginPage
isLoggedIn={isLoggedIn}
setIsLoggedIn={setIsLoggedIn}
register={true}
isLoaded={isLoaded}
globalUrl={globalUrl}
setCookie={setCookie}
cookies={cookies}
checkLogin={checkLogin}
{...props}
/>
}
/>
<Route
exact
path="/admin"
element={
<Admin
userdata={userdata}
isLoggedIn={isLoggedIn}
setIsLoggedIn={setIsLoggedIn}
register={true}
isLoaded={isLoaded}
globalUrl={globalUrl}
setCookie={setCookie}
cookies={cookies}
{...props}
/>
}
/>
<Route
exact
path="/admin/:key"
element={
<Admin
isLoggedIn={isLoggedIn}
setIsLoggedIn={setIsLoggedIn}
register={true}
isLoaded={isLoaded}
globalUrl={globalUrl}
setCookie={setCookie}
cookies={cookies}
{...props}
/>
}
/>
{userdata.id !== undefined ? (
<Route
exact
path="/settings"
element={
<SettingsPage
isLoaded={isLoaded}
setUserData={setUserData}
userdata={userdata}
globalUrl={globalUrl}
{...props}
/>
}
/>
) : null}
<Route
exact
path="/AdminSetup"
element={
<AdminSetup
isLoaded={isLoaded}
userdata={userdata}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/detectionframework"
element={
<FrameworkWrapper
selectedOption={"Draw"}
showOptions={false}
// Dumb for content load (per now), but good for making the site not suddenly reload parts (ajax thingies)
isLoaded={isLoaded}
isLoggedIn={isLoggedIn}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/app"
element={
<FrameworkWrapper
selectedOption={"Draw"}
showOptions={false}
const options = {
timeout: 9000,
position: positions.BOTTOM_LEFT,
};
isLoaded={isLoaded}
isLoggedIn={isLoggedIn}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/usecases"
element={
<Dashboard
isLoaded={isLoaded}
isLoggedIn={isLoggedIn}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/apps/new"
element={
<AppCreator
isLoaded={isLoaded}
isLoggedIn={isLoggedIn}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/apps"
element={
<Apps
isLoaded={isLoaded}
isLoggedIn={isLoggedIn}
globalUrl={globalUrl}
userdata={userdata}
{...props}
/>
}
/>
<Route
exact
path="/apps/edit/:appid"
element={
<AppCreator
isLoaded={isLoaded}
isLoggedIn={isLoggedIn}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/workflows"
element={
<Workflows
cookies={cookies}
removeCookie={removeCookie}
isLoaded={isLoaded}
isLoggedIn={isLoggedIn}
globalUrl={globalUrl}
cookies={cookies}
userdata={userdata}
{...props}
/>
}
/>
<Route
exact
path="/getting-started"
element={
<GettingStarted
cookies={cookies}
removeCookie={removeCookie}
isLoaded={isLoaded}
isLoggedIn={isLoggedIn}
globalUrl={globalUrl}
cookies={cookies}
userdata={userdata}
{...props}
/>
}
/>
<Route
exact
path="/workflows/:key"
element={
<AngularWorkflow
alert={alert}
userdata={userdata}
globalUrl={globalUrl}
isLoaded={isLoaded}
isLoggedIn={isLoggedIn}
{...props}
/>
}
/>
<Route
exact
path="/docs/:key"
element={
<Docs
isMobile={isMobile}
isLoaded={isLoaded}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/docs"
element={
//navigate(`/docs/about`)
<Docs
isMobile={isMobile}
isLoaded={isLoaded}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/support"
element={
//navigate(`/docs/about`)
<Docs
isMobile={isMobile}
isLoaded={isLoaded}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/introduction"
element={
<Introduction
isLoaded={isLoaded}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/introduction/:key"
element={
<Introduction
isLoaded={isLoaded}
globalUrl={globalUrl}
{...props}
/>
}
/>
<Route
exact
path="/set_authentication"
element={
<SetAuthentication
userdata={userdata}
isLoggedIn={isLoggedIn}
setIsLoggedIn={setIsLoggedIn}
register={true}
isLoaded={isLoaded}
globalUrl={globalUrl}
setCookie={setCookie}
cookies={cookies}
{...props}
/>
}
/>
<Route
exact
path="/login_sso"
element={
<SetAuthenticationSSO
userdata={userdata}
isLoggedIn={isLoggedIn}
setIsLoggedIn={setIsLoggedIn}
register={true}
isLoaded={isLoaded}
globalUrl={globalUrl}
setCookie={setCookie}
cookies={cookies}
{...props}
/>
}
/>
<Route
exact
path="/keepalive"
element={
<KeepAlive
isLoggedIn={isLoggedIn}
setIsLoggedIn={setIsLoggedIn}
register={true}
isLoaded={isLoaded}
globalUrl={globalUrl}
setCookie={setCookie}
cookies={cookies}
{...props}
/>
}
/>
<Route
exact
path="/"
element={
<LoginPage
isLoggedIn={isLoggedIn}
setIsLoggedIn={setIsLoggedIn}
register={true}
isLoaded={isLoaded}
globalUrl={globalUrl}
setCookie={setCookie}
cookies={cookies}
{...props}
/>
}
/>
</Routes>
</div>
);
const includedData = window.location.pathname === "/home" || window.location.pathname === "/features" ?
<div>
<Route exact path="/home" render={props => <LandingPageNew isLoaded={isLoaded} {...props} />} />
</div> :
<div style={{ backgroundColor: "#1F2023", color: "rgba(255, 255, 255, 0.65)", minHeight: "100vh" }}>
<Header cookies={cookies} removeCookie={removeCookie} isLoaded={isLoaded} globalUrl={globalUrl} setIsLoggedIn={setIsLoggedIn} isLoggedIn={isLoggedIn} userdata={userdata} {...props} />
<Route exact path="/oauth2" render={props => <Oauth2 isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/contact" render={props => <Contact isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/login" render={props => <LoginPage isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
<Route exact path="/admin" render={props => <Admin userdata={userdata} isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
<Route exact path="/admin/:key" render={props => <Admin isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
<Route exact path="/settings" render={props => <SettingsPage isLoaded={isLoaded} userdata={userdata} globalUrl={globalUrl} {...props} />} />
<Route exact path="/AdminSetup" render={props => <AdminSetup isLoaded={isLoaded} userdata={userdata} globalUrl={globalUrl} {...props} />} />
<Route exact path="/webhooks" render={props => <Webhooks isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/webhooks/:key" render={props => <EditWebhook isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/schedules" render={props => <Schedules globalUrl={globalUrl} {...props} />} />
<Route exact path="/dashboard" render={props => <Dashboard isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} {...props} />} />
<Route exact path="/apps" render={props => <Apps isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} userdata={userdata} {...props} />} />
<Route exact path="/apps/new" render={props => <AppCreator isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} {...props} />} />
<Route exact path="/apps/edit/:appid" render={props => <AppCreator isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} {...props} />} />
<Route exact path="/schedules/:key" render={props => <EditSchedule globalUrl={globalUrl} {...props} />} />
<Route exact path="/workflows" render={props => <Workflows cookies={cookies} removeCookie={removeCookie} isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} cookies={cookies} userdata={userdata} {...props} />} />
<Route exact path="/workflows/:key" render={props => <AngularWorkflow userdata={userdata} globalUrl={globalUrl} isLoaded={isLoaded} isLoggedIn={isLoggedIn} {...props} />} />
<Route exact path="/docs/:key" render={props => <Docs isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/docs" render={props => { window.location.pathname = "/docs/about" }} />
<Route exact path="/introduction" render={props => <Introduction isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/introduction/:key" render={props => <Introduction isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/" render={props => <LoginPage isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
</div>
// <div style={{backgroundColor: "rgba(21, 32, 43, 1)", color: "#fffff", minHeight: "100vh"}}>
// backgroundColor: "#213243",
// This is a mess hahahah
return (
<MuiThemeProvider theme={theme}>
<CookiesProvider>
<BrowserRouter>
<Provider template={AlertTemplate} {...options}>
{includedData}
</Provider>
</BrowserRouter>
</CookiesProvider>
</MuiThemeProvider>
);
// <div style={{backgroundColor: "rgba(21, 32, 43, 1)", color: "#fffff", minHeight: "100vh"}}>
// backgroundColor: "#213243",
// This is a mess hahahah
return (
<MuiThemeProvider theme={theme}>
<CookiesProvider>
<BrowserRouter>
<Provider template={AlertTemplate} {...options}>
{includedData}
</Provider>
</BrowserRouter>
</CookiesProvider>
</MuiThemeProvider>
);
};
export default App;
frontend/src/__test__/appdata.js
+1348 -1
View File
File diff suppressed because one or more lines are too long
frontend/src/__test__/environmentdata.js
+5 -2
View File
@@ -1,3 +1,6 @@
const data = [{"name": "cloud", "type": "cloud"}, {"name": "onprem", "type": "onprem"}]
const data = [
{ name: "cloud", type: "cloud" },
{ name: "onprem", type: "onprem" },
];
export default data;
export default data;
frontend/src/__test__/scheduledata.js
+25 -26
View File
@@ -1,30 +1,29 @@
const Data = {
"src": {
"name": "Get Tickets",
"description": "Get tickets",
"outputparameters": [{
"name": "SymptomDescription",
"schema": {"type": "string"}},
{"name": "DetailedDescription",
"schema": {"type": "string"}},
{"name": "EventSource",
"schema": {"type": "string"}
}]
},
"dst": {
"name": "Create alert",
"description": "Create alert in TheHive",
"inputparameters": [{
"name": "title",
"required": true,
"schema": {"type": "string"}},
{"name": "description",
"required": true,
"schema": {"type": "string"}},
{"name": "source",
"required": true,
"schema": {"type": "string"}
}]}
src: {
name: "Get Tickets",
description: "Get tickets",
outputparameters: [
{
name: "SymptomDescription",
schema: { type: "string" },
},
{ name: "DetailedDescription", schema: { type: "string" } },
{ name: "EventSource", schema: { type: "string" } },
],
},
dst: {
name: "Create alert",
description: "Create alert in TheHive",
inputparameters: [
{
name: "title",
required: true,
schema: { type: "string" },
},
{ name: "description", required: true, schema: { type: "string" } },
{ name: "source", required: true, schema: { type: "string" } },
],
},
};
export default Data;
frontend/src/__test__/webhookdata.js
+12 -12
View File
@@ -1,15 +1,15 @@
const data = {
"id":"8ccf0bec1fde018771ab685d2a40bd52",
"info":{
"url":"",
"name":"testing",
"description":"wut"
},
"transforms":{},
"actions": {},
"type":"webhook",
"status":"uninitialized",
"running":false
}
id: "8ccf0bec1fde018771ab685d2a40bd52",
info: {
url: "",
name: "testing",
description: "wut",
},
transforms: {},
actions: {},
type: "webhook",
status: "uninitialized",
running: false,
};
export default data;
frontend/src/__test__/workflowdata.js
+168 -2
View File
@@ -1,3 +1,169 @@
const data = {"actions":[{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"70574332-da82-cf17-c723-75fa7b8493c2","is_valid":true,"label":"hello_world","environment":"onprem","name":"hello_world","parameters":null,"position":{"x":353.7438792397648,"y":260.6717930890377},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"30522433-56ed-53c3-575d-766e282e1d3e","is_valid":true,"label":"random_number","environment":"cloud","name":"random_number","parameters":null,"position":{"x":458.30040774503794,"y":104.27580103487651},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"5b7ac5b5-9514-02b9-ebe0-998c0843b104","is_valid":false,"label":"hello_world_2","environment":"onprem","name":"hello_world","parameters":null,"position":{"x":414.7256019053981,"y":-140.46450482659628},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"7e6e7a19-4636-cebc-91c4-052a3769a18b","is_valid":true,"label":"hello_world_3","environment":"cloud","name":"hello_world","parameters":null,"position":{"x":83.59752786243806,"y":50.232317715020734},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"edbf927d-5a00-2405-28ed-47982cdf5110","is_valid":true,"label":"hello_world_4","environment":"cloud","name":"hello_world","parameters":null,"position":{"x":-147.30681300186404,"y":89.16690830150289},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"4844a855-1e2b-669d-fc72-5f398321ac5d","is_valid":false,"label":"hello_world_5","environment":"onprem","name":"hello_world","parameters":null,"position":{"x":130.24982593523967,"y":233.8325632286361},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09","is_valid":true,"label":"hello_world_6","environment":"cloud","name":"hello_world","parameters":null,"position":{"x":83.551088005629,"y":-105.15867327274223},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"469d8c2b-52ac-e397-9a29-becccd04aed8","is_valid":true,"label":"hello_world_7","environment":"cloud","name":"hello_world","parameters":null,"position":{"x":314.4987657226086,"y":10.167183586257954},"priority":0}],"branches":[{"destination_id":"30522433-56ed-53c3-575d-766e282e1d3e","id":"4bcb9795-94e6-7d5f-2074-0d5b27784e0b","source_id":"70574332-da82-cf17-c723-75fa7b8493c2"},{"destination_id":"5b7ac5b5-9514-02b9-ebe0-998c0843b104","id":"fe0ab8e4-a535-61cd-3c09-8fd3d8e40769","source_id":"30522433-56ed-53c3-575d-766e282e1d3e"},{"destination_id":"469d8c2b-52ac-e397-9a29-becccd04aed8","id":"8b9ee9bc-b0ab-0bb6-af61-46d4594b2663","source_id":"30522433-56ed-53c3-575d-766e282e1d3e"},{"destination_id":"6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09","id":"c204d5ef-9cc1-d906-9988-86a624c57783","source_id":"469d8c2b-52ac-e397-9a29-becccd04aed8"},{"destination_id":"6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09","id":"1ffb3934-60ec-8f80-5cee-3ddc0a37fdb6","source_id":"5b7ac5b5-9514-02b9-ebe0-998c0843b104"},{"destination_id":"edbf927d-5a00-2405-28ed-47982cdf5110","id":"9c7fb048-9d0d-cb84-9ba0-be729af9b4d1","source_id":"6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09"},{"destination_id":"edbf927d-5a00-2405-28ed-47982cdf5110","id":"e3ab104e-fc8b-3af5-8daa-bfa57bcf9690","source_id":"7e6e7a19-4636-cebc-91c4-052a3769a18b"},{"destination_id":"7e6e7a19-4636-cebc-91c4-052a3769a18b","id":"b6626081-22dd-3af3-b899-480f60d886ca","source_id":"30522433-56ed-53c3-575d-766e282e1d3e"},{"destination_id":"4844a855-1e2b-669d-fc72-5f398321ac5d","id":"4275cf97-0447-bbda-0c80-ab20d389de1a","source_id":"edbf927d-5a00-2405-28ed-47982cdf5110"}],"conditions":[],"triggers":[],"transforms":[],"description":"asd","id":"2f299808-0f1b-4ae0-97fc-ac17483dfcf7","id":"2f299808-0f1b-4ae0-97fc-ac17483dfcf7","is_valid":true,"name":"test2","start":"70574332-da82-cf17-c723-75fa7b8493c2","owner":{"username":"","id":"","orgs":""},"execution_org":{"name":"","org":"","users":null,"id":""},"workflow_variables":null}
const data = {
actions: [
{
app_name: "hello_world",
app_version: "1.0.0",
errors: null,
id: "70574332-da82-cf17-c723-75fa7b8493c2",
is_valid: true,
label: "hello_world",
environment: "onprem",
name: "hello_world",
parameters: null,
position: { x: 353.7438792397648, y: 260.6717930890377 },
priority: 0,
},
{
app_name: "hello_world",
app_version: "1.0.0",
errors: null,
id: "30522433-56ed-53c3-575d-766e282e1d3e",
is_valid: true,
label: "random_number",
environment: "cloud",
name: "random_number",
parameters: null,
position: { x: 458.30040774503794, y: 104.27580103487651 },
priority: 0,
},
{
app_name: "hello_world",
app_version: "1.0.0",
errors: null,
id: "5b7ac5b5-9514-02b9-ebe0-998c0843b104",
is_valid: false,
label: "hello_world_2",
environment: "onprem",
name: "hello_world",
parameters: null,
position: { x: 414.7256019053981, y: -140.46450482659628 },
priority: 0,
},
{
app_name: "hello_world",
app_version: "1.0.0",
errors: null,
id: "7e6e7a19-4636-cebc-91c4-052a3769a18b",
is_valid: true,
label: "hello_world_3",
environment: "cloud",
name: "hello_world",
parameters: null,
position: { x: 83.59752786243806, y: 50.232317715020734 },
priority: 0,
},
{
app_name: "hello_world",
app_version: "1.0.0",
errors: null,
id: "edbf927d-5a00-2405-28ed-47982cdf5110",
is_valid: true,
label: "hello_world_4",
environment: "cloud",
name: "hello_world",
parameters: null,
position: { x: -147.30681300186404, y: 89.16690830150289 },
priority: 0,
},
{
app_name: "hello_world",
app_version: "1.0.0",
errors: null,
id: "4844a855-1e2b-669d-fc72-5f398321ac5d",
is_valid: false,
label: "hello_world_5",
environment: "onprem",
name: "hello_world",
parameters: null,
position: { x: 130.24982593523967, y: 233.8325632286361 },
priority: 0,
},
{
app_name: "hello_world",
app_version: "1.0.0",
errors: null,
id: "6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09",
is_valid: true,
label: "hello_world_6",
environment: "cloud",
name: "hello_world",
parameters: null,
position: { x: 83.551088005629, y: -105.15867327274223 },
priority: 0,
},
{
app_name: "hello_world",
app_version: "1.0.0",
errors: null,
id: "469d8c2b-52ac-e397-9a29-becccd04aed8",
is_valid: true,
label: "hello_world_7",
environment: "cloud",
name: "hello_world",
parameters: null,
position: { x: 314.4987657226086, y: 10.167183586257954 },
priority: 0,
},
],
branches: [
{
destination_id: "30522433-56ed-53c3-575d-766e282e1d3e",
id: "4bcb9795-94e6-7d5f-2074-0d5b27784e0b",
source_id: "70574332-da82-cf17-c723-75fa7b8493c2",
},
{
destination_id: "5b7ac5b5-9514-02b9-ebe0-998c0843b104",
id: "fe0ab8e4-a535-61cd-3c09-8fd3d8e40769",
source_id: "30522433-56ed-53c3-575d-766e282e1d3e",
},
{
destination_id: "469d8c2b-52ac-e397-9a29-becccd04aed8",
id: "8b9ee9bc-b0ab-0bb6-af61-46d4594b2663",
source_id: "30522433-56ed-53c3-575d-766e282e1d3e",
},
{
destination_id: "6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09",
id: "c204d5ef-9cc1-d906-9988-86a624c57783",
source_id: "469d8c2b-52ac-e397-9a29-becccd04aed8",
},
{
destination_id: "6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09",
id: "1ffb3934-60ec-8f80-5cee-3ddc0a37fdb6",
source_id: "5b7ac5b5-9514-02b9-ebe0-998c0843b104",
},
{
destination_id: "edbf927d-5a00-2405-28ed-47982cdf5110",
id: "9c7fb048-9d0d-cb84-9ba0-be729af9b4d1",
source_id: "6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09",
},
{
destination_id: "edbf927d-5a00-2405-28ed-47982cdf5110",
id: "e3ab104e-fc8b-3af5-8daa-bfa57bcf9690",
source_id: "7e6e7a19-4636-cebc-91c4-052a3769a18b",
},
{
destination_id: "7e6e7a19-4636-cebc-91c4-052a3769a18b",
id: "b6626081-22dd-3af3-b899-480f60d886ca",
source_id: "30522433-56ed-53c3-575d-766e282e1d3e",
},
{
destination_id: "4844a855-1e2b-669d-fc72-5f398321ac5d",
id: "4275cf97-0447-bbda-0c80-ab20d389de1a",
source_id: "edbf927d-5a00-2405-28ed-47982cdf5110",
},
],
conditions: [],
triggers: [],
transforms: [],
description: "asd",
id: "2f299808-0f1b-4ae0-97fc-ac17483dfcf7",
id: "2f299808-0f1b-4ae0-97fc-ac17483dfcf7",
is_valid: true,
name: "test2",
start: "70574332-da82-cf17-c723-75fa7b8493c2",
owner: { username: "", id: "", orgs: "" },
execution_org: { name: "", org: "", users: null, id: "" },
workflow_variables: null,
};
export default data;
export default data;
frontend/src/assets/img/bag.svg
+9
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 14 KiB

frontend/src/assets/img/book.svg
+9
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

frontend/src/assets/img/github_phishing_email.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 449 KiB

frontend/src/assets/img/github_shuffle_img.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 431 KiB

frontend/src/assets/img/mobile.svg
+9
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

frontend/src/charts.js
+68 -68
View File
@@ -23,7 +23,7 @@
let chart1_2_options = {
maintainAspectRatio: false,
legend: {
display: false
display: false,
},
tooltips: {
backgroundColor: "#f5f5f5",
@@ -33,7 +33,7 @@ let chart1_2_options = {
xPadding: 12,
mode: "nearest",
intersect: 0,
position: "nearest"
position: "nearest",
},
responsive: true,
scales: {
@@ -43,15 +43,15 @@ let chart1_2_options = {
gridLines: {
drawBorder: false,
color: "rgba(29,140,248,0.0)",
zeroLineColor: "transparent"
zeroLineColor: "transparent",
},
ticks: {
suggestedMin: 60,
suggestedMax: 125,
padding: 20,
fontColor: "#9a9a9a"
}
}
fontColor: "#9a9a9a",
},
},
],
xAxes: [
{
@@ -59,22 +59,22 @@ let chart1_2_options = {
gridLines: {
drawBorder: false,
color: "rgba(29,140,248,0.1)",
zeroLineColor: "transparent"
zeroLineColor: "transparent",
},
ticks: {
padding: 20,
fontColor: "#9a9a9a"
}
}
]
}
fontColor: "#9a9a9a",
},
},
],
},
};
// #########################################
// // // used inside src/views/Dashboard.js
// #########################################
let chartExample1 = {
data1: canvas => {
data1: (canvas) => {
let ctx = canvas.getContext("2d");
let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -96,7 +96,7 @@ let chartExample1 = {
"SEP",
"OCT",
"NOV",
"DEC"
"DEC",
],
datasets: [
{
@@ -114,12 +114,12 @@ let chartExample1 = {
pointHoverRadius: 4,
pointHoverBorderWidth: 15,
pointRadius: 4,
data: [100, 70, 90, 70, 85, 60, 75, 60, 90, 80, 110, 100]
}
]
data: [100, 70, 90, 70, 85, 60, 75, 60, 90, 80, 110, 100],
},
],
};
},
data2: canvas => {
data2: (canvas) => {
let ctx = canvas.getContext("2d");
let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -141,7 +141,7 @@ let chartExample1 = {
"SEP",
"OCT",
"NOV",
"DEC"
"DEC",
],
datasets: [
{
@@ -159,12 +159,12 @@ let chartExample1 = {
pointHoverRadius: 4,
pointHoverBorderWidth: 15,
pointRadius: 4,
data: [80, 120, 105, 110, 95, 105, 90, 100, 80, 95, 70, 120]
}
]
data: [80, 120, 105, 110, 95, 105, 90, 100, 80, 95, 70, 120],
},
],
};
},
data3: canvas => {
data3: (canvas) => {
let ctx = canvas.getContext("2d");
let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -186,7 +186,7 @@ let chartExample1 = {
"SEP",
"OCT",
"NOV",
"DEC"
"DEC",
],
datasets: [
{
@@ -204,19 +204,19 @@ let chartExample1 = {
pointHoverRadius: 4,
pointHoverBorderWidth: 15,
pointRadius: 4,
data: [60, 80, 65, 130, 80, 105, 90, 130, 70, 115, 60, 130]
}
]
data: [60, 80, 65, 130, 80, 105, 90, 130, 70, 115, 60, 130],
},
],
};
},
options: chart1_2_options
options: chart1_2_options,
};
// #########################################
// // // used inside src/views/Dashboard.js
// #########################################
let chartExample2 = {
data: canvas => {
data: (canvas) => {
let ctx = canvas.getContext("2d");
let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -243,19 +243,19 @@ let chartExample2 = {
pointHoverRadius: 4,
pointHoverBorderWidth: 15,
pointRadius: 4,
data: [80, 100, 70, 80, 120, 80]
}
]
data: [80, 100, 70, 80, 120, 80],
},
],
};
},
options: chart1_2_options
options: chart1_2_options,
};
// #########################################
// // // used inside src/views/Dashboard.js
// #########################################
let chartExample3 = {
data: canvas => {
data: (canvas) => {
let ctx = canvas.getContext("2d");
let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -276,15 +276,15 @@ let chartExample3 = {
borderWidth: 2,
borderDash: [],
borderDashOffset: 0.0,
data: [53, 20, 10, 80, 100, 45]
}
]
data: [53, 20, 10, 80, 100, 45],
},
],
};
},
options: {
maintainAspectRatio: false,
legend: {
display: false
display: false,
},
tooltips: {
backgroundColor: "#f5f5f5",
@@ -294,7 +294,7 @@ let chartExample3 = {
xPadding: 12,
mode: "nearest",
intersect: 0,
position: "nearest"
position: "nearest",
},
responsive: true,
scales: {
@@ -303,38 +303,38 @@ let chartExample3 = {
gridLines: {
drawBorder: false,
color: "rgba(225,78,202,0.1)",
zeroLineColor: "transparent"
zeroLineColor: "transparent",
},
ticks: {
suggestedMin: 60,
suggestedMax: 120,
padding: 20,
fontColor: "#9e9e9e"
}
}
fontColor: "#9e9e9e",
},
},
],
xAxes: [
{
gridLines: {
drawBorder: false,
color: "rgba(225,78,202,0.1)",
zeroLineColor: "transparent"
zeroLineColor: "transparent",
},
ticks: {
padding: 20,
fontColor: "#9e9e9e"
}
}
]
}
}
fontColor: "#9e9e9e",
},
},
],
},
},
};
// #########################################
// // // used inside src/views/Dashboard.js
// #########################################
const chartExample4 = {
data: canvas => {
data: (canvas) => {
let ctx = canvas.getContext("2d");
let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -361,15 +361,15 @@ const chartExample4 = {
pointHoverRadius: 4,
pointHoverBorderWidth: 15,
pointRadius: 4,
data: [90, 27, 60, 12, 80]
}
]
data: [90, 27, 60, 12, 80],
},
],
};
},
options: {
maintainAspectRatio: false,
legend: {
display: false
display: false,
},
tooltips: {
@@ -380,7 +380,7 @@ const chartExample4 = {
xPadding: 12,
mode: "nearest",
intersect: 0,
position: "nearest"
position: "nearest",
},
responsive: true,
scales: {
@@ -390,15 +390,15 @@ const chartExample4 = {
gridLines: {
drawBorder: false,
color: "rgba(29,140,248,0.0)",
zeroLineColor: "transparent"
zeroLineColor: "transparent",
},
ticks: {
suggestedMin: 50,
suggestedMax: 125,
padding: 20,
fontColor: "#9e9e9e"
}
}
fontColor: "#9e9e9e",
},
},
],
xAxes: [
@@ -407,21 +407,21 @@ const chartExample4 = {
gridLines: {
drawBorder: false,
color: "rgba(0,242,195,0.1)",
zeroLineColor: "transparent"
zeroLineColor: "transparent",
},
ticks: {
padding: 20,
fontColor: "#9e9e9e"
}
}
]
}
}
fontColor: "#9e9e9e",
},
},
],
},
},
};
module.exports = {
chartExample1, // in src/views/Dashboard.js
chartExample2, // in src/views/Dashboard.js
chartExample3, // in src/views/Dashboard.js
chartExample4 // in src/views/Dashboard.js
chartExample4, // in src/views/Dashboard.js
};
frontend/src/components/AlertPopup.js
+13 -20
View File
@@ -1,26 +1,19 @@
import React, { useEffect} from 'react';
import React, { useEffect } from "react";
const Popup = (props) => {
const { data } = props;
const { data } = props;
const popupStyle = {
position: "fixed",
width: "300px",
height: "50px",
backgroundColor: "black",
color: "white",
}
const popupStyle = {
position: "fixed",
width: "300px",
height: "50px",
backgroundColor: "black",
color: "white",
};
const popupData =
<div>
HEY
</div>
const popupData = <div>HEY</div>;
return (
<div>
{popupData}
</div>
)
}
return <div>{popupData}</div>;
};
export default Popup
export default Popup;
frontend/src/components/AlertTemplate.js
+34 -31
View File
@@ -1,45 +1,48 @@
import React from 'react'
import InfoIcon from '@material-ui/icons/Info';
import CheckIcon from '@material-ui/icons/Check';
import ErrorOutlineIcon from '@material-ui/icons/ErrorOutline';
import CloseIcon from '@material-ui/icons/Close';
import Typography from '@material-ui/core/Typography';
import React from "react";
import InfoIcon from "@material-ui/icons/Info";
import CheckIcon from "@material-ui/icons/Check";
import ErrorOutlineIcon from "@material-ui/icons/ErrorOutline";
import CloseIcon from "@material-ui/icons/Close";
import Typography from "@material-ui/core/Typography";
const alertStyle = {
backgroundColor: 'rgba(0,0,0,0.9)',
color: 'white',
backgroundColor: "rgba(0,0,0,0.9)",
color: "white",
padding: 15,
textTransform: 'uppercase',
borderRadius: '3px',
display: 'flex',
justifyContent: 'space-between',
alignItems: 'center',
boxShadow: '0px 2px 2px 2px rgba(0, 0, 0, 0.03)',
width: 400,
boxSizing: 'border-box',
zIndex: 100001,
}
textTransform: "uppercase",
borderRadius: "3px",
display: "flex",
justifyContent: "space-between",
alignItems: "center",
boxShadow: "0px 2px 2px 2px rgba(0, 0, 0, 0.03)",
width: 300,
boxSizing: "border-box",
zIndex: 100001,
overflow: "hidden",
};
const buttonStyle = {
marginLeft: '20px',
border: 'none',
backgroundColor: 'transparent',
cursor: 'pointer',
color: '#FFFFFF'
}
marginLeft: "20px",
border: "none",
backgroundColor: "transparent",
cursor: "pointer",
color: "#FFFFFF",
};
const AlertTemplate = ({ message, options, style, close }) => {
return (
<div style={{ ...alertStyle, ...style }}>
{options.type === 'info' && <InfoIcon style={{color: "white"}} />}
{options.type === 'success' && <CheckIcon style={{color: "green", }}/>}
{options.type === 'error' && <ErrorOutlineIcon style={{color: "red"}} />}
<Typography style={{marginLeft: 15, flex: 2 }}>{message}</Typography>
{options.type === "info" && <InfoIcon style={{ color: "white" }} />}
{options.type === "success" && <CheckIcon style={{ color: "green" }} />}
{options.type === "error" && (
<ErrorOutlineIcon style={{ color: "red" }} />
)}
<Typography style={{ marginLeft: 15, flex: 2 }}>{message}</Typography>
<button onClick={close} style={buttonStyle}>
<CloseIcon />
</button>
</div>
)
}
);
};
export default AlertTemplate
export default AlertTemplate;
frontend/src/components/ConfigureWorkflow.jsx
+677
View File
@@ -0,0 +1,677 @@
import React, { useState } from "react";
import {
InputAdornment,
Tooltip,
TextField,
CircularProgress,
ButtonGroup,
Button,
Avatar,
ListItemAvatar,
Typography,
List,
ListItem,
ListItemText,
} from "@material-ui/core";
import { FavoriteBorder as FavoriteBorderIcon } from "@material-ui/icons";
import { FixName } from "../views/Apps.jsx";
// Handles workflow updates on first open to highlight the issues of the workflow
// Variables
// Action (exists, missing fields)
// Action auth
// Triggers
//
// Specifically used for UNSAVED workflows only?
const ConfigureWorkflow = (props) => {
const {
globalUrl,
theme,
workflow,
appAuthentication,
setSelectedAction,
setAuthenticationModalOpen,
setSelectedApp,
apps,
selectedAction,
setConfigureWorkflowModalOpen,
saveWorkflow,
newWebhook,
submitSchedule,
referenceUrl,
isCloud,
setAuthenticationType,
alert,
} = props;
const [requiredActions, setRequiredActions] = React.useState([]);
const [requiredVariables, setRequiredVariables] = React.useState([]);
const [requiredTriggers, setRequiredTriggers] = React.useState([]);
const [previousAuth, setPreviousAuth] = React.useState(appAuthentication);
const [firstLoad, setFirstLoad] = React.useState("");
const [itemChanged, setItemChanged] = React.useState(false);
var finished = false;
if (workflow === undefined || workflow === null) {
return null;
}
if (apps === undefined || apps === null) {
return null;
}
if (appAuthentication === undefined || appAuthentication === null) {
return null;
}
const getApp = (actionId, appId) => {
fetch(globalUrl + "/api/v1/apps/" + appId + "/config?openapi=false", {
headers: {
Accept: "application/json",
},
credentials: "include",
})
.then((response) => {
if (response.status === 200) {
//alert.success("Successfully GOT app "+appId)
} else {
alert.error("Failed getting app");
}
return response.json();
})
.then((responseJson) => {
console.log("ACTION: ", responseJson);
if (
responseJson.actions !== undefined &&
responseJson.actions !== null
) {
}
})
.catch((error) => {
alert.error(error.toString());
});
};
if (firstLoad.length === 0 || firstLoad !== workflow.id) {
if (finished) {
setConfigureWorkflowModalOpen(false);
return null;
}
if (apps === undefined || apps === null || apps.length === 0) {
console.log("No apps loaded: ", apps);
setConfigureWorkflowModalOpen(false);
return null;
}
setFirstLoad(workflow.id);
const newactions = [];
for (var key in workflow.actions) {
const action = workflow.actions[key];
var newaction = {
large_image: action.large_image,
app_name: action.app_name,
app_version: action.app_version,
activation_done: false,
must_activate: false,
must_authenticate: false,
auth_done: false,
action_ids: [],
action: action,
update_version: action.app_version,
app: {},
};
const app = apps.find(
(app) =>
app.name === action.app_name &&
(app.app_version === action.app_version ||
(app.loop_versions !== null &&
app.loop_versions.includes(action.app_version)))
);
if (app === undefined || app === null) {
//console.log("App not found: ", action.app_name);
const subapp = apps.find(app => app.name === action.app_name)
if (subapp !== undefined && subapp !== null) {
newaction.update_version = "1.1.0"
}
newaction.must_activate = true;
} else {
if (
action.authentication_id === "" &&
app.authentication.required === true
) {
// Check if configuration is filled or not
var filled = true;
for (var key in action.parameters) {
if (action.parameters[key].configuration) {
//console.log("Found config: ", action.parameters[key])
if (
action.parameters[key].value === null ||
action.parameters[key].value.length === 0
) {
filled = false;
break;
}
}
}
if (!filled) {
newaction.must_authenticate = true;
newaction.action_ids.push(action.id);
}
} else if (action.authentication_id !== "" && app.authentication.required === true) {
console.log("Should verify authentication ID ", action.authentication_id)
}
newaction.app = app;
}
if (
action.errors !== undefined &&
action.errors !== null &&
action.errors.length > 0
) {
//console.log("Node has errors!: ", action.errors)
}
if (newaction.must_authenticate) {
var authenticationOptions = [];
for (var key in appAuthentication) {
const auth = appAuthentication[key];
if (auth.app.name === app.name && auth.active) {
//console.log("Found auth: ", auth)
authenticationOptions.push(auth);
newaction.authenticationId = auth.id;
break;
}
}
if (
newaction.authenticationId === null ||
newaction.authenticationId === undefined ||
newaction.authenticationId.length === ""
) {
//console.log("FAILED to authenticate node!")
if (
newactions.find(
(tmpaction) =>
tmpaction.app_id === newaction.app_id &&
tmpaction.app_name === newaction.app_name
) !== undefined
) {
//console.log("Action already found.");
} else {
newactions.push(newaction);
}
} else {
//console.log("Skipping node as it's already authenticated.")
newaction.authentication = authenticationOptions;
workflow.actions[key] = newaction;
}
} else if (newaction.must_activate) {
if (
newactions.find(
(tmpaction) =>
tmpaction.app_id === newaction.app_id &&
tmpaction.app_name === newaction.app_name
) !== undefined
) {
console.log("Action already found.");
} else {
newactions.push(newaction);
}
}
}
for (var key in workflow.workflow_variables) {
const variable = workflow.workflow_variables[key];
if (
variable.value === undefined ||
variable.value === undefined ||
variable.value.length < 2
) {
variable.value = "";
variable.index = key;
requiredVariables.push(variable);
}
}
for (var key in workflow.triggers) {
var trigger = workflow.triggers[key];
trigger.index = key;
if (trigger.status === "running") {
continue;
}
if (
trigger.trigger_type === "SUBFLOW" ||
trigger.trigger_type === "USERINPUT"
) {
continue;
}
requiredTriggers.push(trigger);
}
if (
requiredTriggers.length === 0 &&
requiredVariables.length === 0 &&
newactions.length === 0
) {
setConfigureWorkflowModalOpen(false);
}
setRequiredTriggers(requiredTriggers);
setRequiredVariables(requiredVariables);
setRequiredActions(newactions);
}
if (appAuthentication.length !== previousAuth.length) {
var newactions = [];
for (var actionkey in requiredActions) {
var newaction = requiredActions[actionkey];
const app = newaction.app;
for (var key in appAuthentication) {
const auth = appAuthentication[key];
if (auth.app.name === app.name && auth.active) {
newaction.auth_done = true;
break;
}
}
newactions.push(newaction);
}
setRequiredActions(newactions);
setPreviousAuth(appAuthentication);
// Set auth done to true
//"auth_done": false
}
const TriggerSection = (props) => {
const { trigger } = props;
return (
<ListItem>
<ListItemAvatar>
<Avatar variant="rounded">
<img
alt={trigger.label}
src={trigger.large_image}
style={{ width: 50 }}
/>
</Avatar>
</ListItemAvatar>
<ListItemText
primary={trigger.name}
secondary={trigger.description}
style={{}}
/>
{trigger.trigger_type === "WEBHOOK" && trigger.status !== "running" ? (
<Button
disabled={trigger.status === "running"}
color="primary"
variant="contained"
onClick={() => {
workflow.triggers[trigger.index].status = "running";
if (workflow.triggers[trigger.index].parameters === null) {
workflow.triggers[trigger.index].parameters = [
{
name: "url",
value: referenceUrl + "webhook_" + trigger.id,
},
{ name: "tmp", value: "webhook_" + trigger.id },
];
}
newWebhook(workflow.triggers[trigger.index]);
saveWorkflow(workflow);
setItemChanged(true);
}}
>
{trigger.status !== "running" ? "Start" : "Running"}
</Button>
) : trigger.trigger_type === "SCHEDULE" &&
trigger.status !== "running" ? (
<Button
disabled={trigger.status === "running"}
color="primary"
variant="contained"
onClick={() => {
workflow.triggers[trigger.index].status = "running";
if (workflow.triggers[trigger.index].parameters === null) {
workflow.triggers[trigger.index].parameters = [
{ name: "cron", value: isCloud ? "*/15 * * * *" : "120" },
{
name: "execution_argument",
value: '{"example": {"json": "is cool"}}',
},
];
}
submitSchedule(workflow.triggers[trigger.index], trigger.index);
saveWorkflow(workflow);
setItemChanged(true);
}}
>
{trigger.status !== "running" ? "Start" : "Running"}
</Button>
) : null}
{/*
<ListItemText
primary={
<TextField
style={{backgroundColor: theme.palette.inputColor, borderRadius: 5,}}
InputProps={{
style:{
color: "white",
minHeight: 50,
marginLeft: 5,
maxWidth: "95%",
fontSize: "1em",
},
endAdornment: (
<InputAdornment position="end">
</InputAdornment>
)
}}
fullWidth
color="primary"
type={"text"}
placeholder={`New value for ${trigger.name}`}
onChange={(event) => {
console.log("NEW VALUE ON INDEX", trigger.value)
}}
onBlur={(event) => {
//workflow.variables[variable.index] = event.target.value
}}
/>
}
style={{}}
/>
*/}
</ListItem>
);
};
const VariableSection = (props) => {
const { variable } = props;
//<Typography variant="body2">Name: {variable.name} - {variable.value}. </Typography>
return (
<ListItem>
<ListItemAvatar>
<Avatar>
<FavoriteBorderIcon />
</Avatar>
</ListItemAvatar>
<ListItemText
primary={variable.name}
secondary={variable.description}
style={{}}
/>
<ListItemText
primary={
<TextField
style={{
backgroundColor: theme.palette.inputColor,
borderRadius: 5,
}}
InputProps={{
style: {
color: "white",
minHeight: 50,
marginLeft: 5,
maxWidth: "95%",
fontSize: "1em",
},
endAdornment: <InputAdornment position="end"></InputAdornment>,
}}
fullWidth
color="primary"
type={"text"}
placeholder={`New value for ${variable.name}`}
onChange={(event) => {
console.log(
"NEW VALUE ON INDEX",
variable.index,
variable.value
);
}}
onBlur={(event) => {
workflow.workflow_variables[variable.index].value =
event.target.value;
}}
/>
}
style={{}}
/>
</ListItem>
);
};
const activateApp = (app_id, app_name, app_version) => {
fetch(
`${globalUrl}/api/v1/apps/${app_id}/activate?app_name=${app_name}&app_version=${app_version}`,
{
method: "GET",
headers: {
"Content-Type": "application/json",
Accept: "application/json",
},
credentials: "include",
}
)
.then((response) => {
if (response.status !== 200) {
//window.location.pathname = "/search"
//alert.error("Failed to find this app. Is it public?")
}
return response.json();
})
.then((responseJson) => {
if (responseJson.success === false) {
if (responseJson.reason !== undefined) {
alert.error("Failed to activate the app: "+responseJson.reason);
} else {
alert.error("Failed to activate the app");
}
} else {
alert.success("App activated for your organization!");
}
})
.catch((error) => {
alert.error(error.toString());
});
};
const AppSection = (props) => {
const { action } = props;
return (
<ListItem>
<ListItemAvatar>
<Avatar variant="rounded">
<img
alt={action.app_name}
src={action.large_image}
style={{ width: 50 }}
/>
</Avatar>
</ListItemAvatar>
<ListItemText
primary={FixName(action.app_name)}
secondary={action.app_version}
style={{}}
/>
{action.must_authenticate ? (
action.auth_done ? (
<Button color="primary" variant="outlined" onClick={() => {}}>
Authenticated
</Button>
) : selectedAction.app_name === action.app_name ? (
<CircularProgress />
) : (
<Button
color="primary"
variant="contained"
onClick={() => {
setAuthenticationType(
action.app.authentication.type === "oauth2" &&
action.app.authentication.redirect_uri !== undefined &&
action.app.authentication.redirect_uri !== null
? {
type: "oauth2",
redirect_uri: action.app.authentication.redirect_uri,
token_uri: action.app.authentication.token_uri,
scope: action.app.authentication.scope,
}
: {
type: "",
}
);
setItemChanged(true);
setSelectedAction(action.action);
setSelectedApp(action.app);
setAuthenticationModalOpen(true);
}}
>
Authenticate
</Button>
)
) : null}
{action.must_activate ? (
<Button
color="primary"
variant="contained"
onClick={() => {
console.log("ACTION: ", action)
activateApp(action.action.app_id, action.app_name, action.app_version);
setItemChanged(true);
}}
>
Activate
</Button>
) : null}
{action.update_version !== action.app_version ? (
<Button
color="primary"
variant="contained"
style={{marginLeft: 5}}
onClick={() => {
console.log("Set version to: ", action.update_version)
if (workflow.actions !== null) {
//console.log(workflow.actions)
alert.info("Setting action to version "+action.update_version)
for (var key in workflow.actions) {
if (workflow.actions[key].app_name === action.app_name && workflow.actions[key].app_version === action.app_version) {
workflow.actions[key].app_version = action.update_version
if (!itemChanged) {
setItemChanged(true)
}
}
}
action.must_activate = false
action.update_version = action.app_version
}
}}
>
{action.update_version}
</Button>
) : null}
</ListItem>
);
};
return (
<div>
<Typography variant="h6">{workflow.name}</Typography>
<Typography variant="body1" color="textSecondary">
The following configuration makes the workflow ready immediately.
</Typography>
{requiredActions.length > 0 ? (
<span>
<Typography variant="body1" style={{ marginTop: 10 }}>
Actions
</Typography>
<List>
{requiredActions.map((data, index) => {
return <AppSection key={index} action={data} />;
})}
</List>
</span>
) : null}
{requiredVariables.length > 0 ? (
<span>
<Typography variant="body1" style={{ marginTop: 10 }}>
Variables
</Typography>
<List>
{requiredVariables.map((data, index) => {
return <VariableSection key={index} variable={data} />;
})}
</List>
</span>
) : null}
{requiredTriggers.length > 0 ? (
<span>
<Typography variant="body1" style={{ marginTop: 10 }}>
Triggers
</Typography>
<List>
{requiredTriggers.map((data, index) => {
return <TriggerSection key={index} trigger={data} />;
})}
</List>
</span>
) : null}
<div style={{ textAlign: "center", display: "flex", marginTop: 20 }}>
<ButtonGroup style={{ margin: "auto" }}>
{/*
<Button color="primary" variant={"outlined"} style={{
}} onClick={() => {
setConfigureWorkflowModalOpen(false)
}}>
Skip
</Button>
*/}
<Button
color="primary"
variant={itemChanged ? "contained" : "outlined"}
style={{}}
onClick={() => {
if (itemChanged) {
saveWorkflow(workflow);
window.location.reload();
} else {
setConfigureWorkflowModalOpen(false);
}
}}
>
Close window
</Button>
</ButtonGroup>
</div>
</div>
);
};
export default ConfigureWorkflow;
frontend/src/components/DetectionFramework.jsx
+1652
View File
File diff suppressed because it is too large Load Diff
frontend/src/components/Dropzone.js
+22 -22
View File
@@ -1,19 +1,19 @@
import React, { useRef, useState } from 'react';
import { useEffect } from 'react';
import BackupIcon from '@material-ui/icons/Backup';
import React, { useRef, useState } from "react";
import { useEffect } from "react";
import BackupIcon from "@material-ui/icons/Backup";
const dragOverStyle = {
backgroundColor: 'rgba(0,0,0,0.8)',
border: '5px dashed white',
borderRadius: '8px',
width: '100%',
height: '100%',
position: 'absolute',
overflow: 'hidden',
backgroundColor: "rgba(0,0,0,0.8)",
border: "5px dashed white",
borderRadius: "8px",
width: "100%",
height: "100%",
position: "absolute",
overflow: "hidden",
zIndex: 100,
display: 'flex',
alignItems: 'center',
justifyContent: 'center'
display: "flex",
alignItems: "center",
justifyContent: "center",
};
const Dropzone = ({ children, style, onDrop }) => {
@@ -56,21 +56,21 @@ const Dropzone = ({ children, style, onDrop }) => {
useEffect(() => {
if (!dropzoneRef.current) return;
dropzoneRef.current.addEventListener('dragover', handleDragOver);
dropzoneRef.current.addEventListener('dragenter', handleDragEnter);
dropzoneRef.current.addEventListener('dragleave', handleDragLeave);
dropzoneRef.current.addEventListener('drop', handleDrop);
dropzoneRef.current.addEventListener("dragover", handleDragOver);
dropzoneRef.current.addEventListener("dragenter", handleDragEnter);
dropzoneRef.current.addEventListener("dragleave", handleDragLeave);
dropzoneRef.current.addEventListener("drop", handleDrop);
return () => {
dropzoneRef.current.removeEventListener('dragover', handleDragOver);
dropzoneRef.current.removeEventListener('dragenter', handleDragEnter);
dropzoneRef.current.removeEventListener('dragleave', handleDragLeave);
dropzoneRef.current.removeEventListener('drop', handleDrop);
dropzoneRef.current.removeEventListener("dragover", handleDragOver);
dropzoneRef.current.removeEventListener("dragenter", handleDragEnter);
dropzoneRef.current.removeEventListener("dragleave", handleDragLeave);
dropzoneRef.current.removeEventListener("drop", handleDrop);
};
}, [dropzoneRef]);
return (
<div ref={dropzoneRef} style={{ position: 'relative', ...style }}>
<div ref={dropzoneRef} style={{ position: "relative", ...style }}>
{dragging && (
<div style={dragOverStyle}>
<BackupIcon fontSize="large" />
frontend/src/components/FAQ.jsx
+23
View File
@@ -0,0 +1,23 @@
import React, {useState} from 'react';
const FAQItem = (props) => {
const { question, answer } = props
const [isExpanded, setIsExpanded] = useState(false)
return (
<Paper onClick={() => {
setIsExpanded(!isExpanded)
}}>
<Typography variant="body1">
{question}
</Typography>
<Typography variant="body2" color="textSecondary">
{answer}
</Typography>
</Paper>
)
}
export default FAQItem;
frontend/src/components/FooterNew.js
+37 -37
View File
@@ -1,54 +1,54 @@
import React from 'react';
import React from "react";
//import List from '@material-ui/core/List';
//import ListItem from '@material-ui/core/ListItem';
//borderTop: "1px solid #385F71"
const FooterStyle = {
right: "0",
left: "0",
bottom: "0",
height: "130px",
backgroundColor: 'rgba(15, 14, 31, 1)',
right: "0",
left: "0",
bottom: "0",
height: "130px",
backgroundColor: "rgba(15, 14, 31, 1)",
};
const FooterInfo = {
maxWidth: '1150px',
minWidth: '768px',
textAlign: 'center',
margin: 'auto',
maxWidth: "1150px",
minWidth: "768px",
textAlign: "center",
margin: "auto",
};
const hrefStyle = {
color: "#bdbdbd",
textDecoration: "none"
}
const Footer = props => {
return (
<div style={FooterStyle}>
<div style={FooterInfo}>
<Box />
</div>
</div>
);
color: "#bdbdbd",
textDecoration: "none",
};
const Box = props => {
return(
<div style={{display: "flex"}}>
<div style={{flex: "1"}}>
<a style={hrefStyle} href="/about">
<h1>About</h1>
</a>
</div>
<div style={{flex: "1"}}>
<a style={hrefStyle} href="/privacy-policy">
<h1>Privacy Policy</h1>
</a>
</div>
</div>
);
const Footer = (props) => {
return (
<div style={FooterStyle}>
<div style={FooterInfo}>
<Box />
</div>
</div>
);
};
const Box = (props) => {
return (
<div style={{ display: "flex" }}>
<div style={{ flex: "1" }}>
<a style={hrefStyle} href="/about">
<h1>About</h1>
</a>
</div>
<div style={{ flex: "1" }}>
<a style={hrefStyle} href="/privacy-policy">
<h1>Privacy Policy</h1>
</a>
</div>
</div>
);
};
export default Footer;
frontend/src/components/FrameworkData.jsx
+28
View File
File diff suppressed because one or more lines are too long
frontend/src/components/Header.js
+816 -294
View File
File diff suppressed because it is too large Load Diff
frontend/src/components/LandingpageUsecases.jsx
+245
View File
@@ -0,0 +1,245 @@
import React, { useState, useEffect } from 'react';
import {isMobile} from "react-device-detect";
import DetectionFramework, { usecases } from "../components/DetectionFramework.jsx";
import {Link} from 'react-router-dom';
import ReactGA from 'react-ga';
import { Button, LinearProgress, Typography } from '@material-ui/core';
export const securityFramework = [
{
image: <path d="M15.6408 8.39233H18.0922V10.0287H15.6408V8.39233ZM0.115234 8.39233H2.56663V10.0287H0.115234V8.39233ZM9.92083 0.21051V2.66506H8.28656V0.21051H9.92083ZM3.31839 2.25596L5.05889 4.00687L3.89856 5.16051L2.15807 3.42596L3.31839 2.25596ZM13.1485 3.99869L14.8808 2.25596L16.0493 3.42596L14.3088 5.16051L13.1485 3.99869ZM9.10369 4.30142C10.404 4.30142 11.651 4.81863 12.5705 5.73926C13.4899 6.65989 14.0065 7.90854 14.0065 9.21051C14.0065 11.0269 13.0178 12.6141 11.5551 13.4651V14.9378C11.5551 15.1548 11.469 15.3629 11.3158 15.5163C11.1625 15.6698 10.9547 15.756 10.738 15.756H7.46943C7.25271 15.756 7.04487 15.6698 6.89163 15.5163C6.73839 15.3629 6.6523 15.1548 6.6523 14.9378V13.4651C5.18963 12.6141 4.2009 11.0269 4.2009 9.21051C4.2009 7.90854 4.71744 6.65989 5.63689 5.73926C6.55635 4.81863 7.80339 4.30142 9.10369 4.30142ZM10.738 16.5741V17.3923C10.738 17.6093 10.6519 17.8174 10.4986 17.9709C10.3454 18.1243 10.1375 18.2105 9.92083 18.2105H8.28656C8.06984 18.2105 7.862 18.1243 7.70876 17.9709C7.55552 17.8174 7.46943 17.6093 7.46943 17.3923V16.5741H10.738ZM8.28656 14.1196H9.92083V12.3769C11.3345 12.0169 12.3722 10.7323 12.3722 9.21051C12.3722 8.34253 12.0279 7.5101 11.4149 6.89634C10.8019 6.28259 9.97056 5.93778 9.10369 5.93778C8.23683 5.93778 7.40546 6.28259 6.79249 6.89634C6.17953 7.5101 5.83516 8.34253 5.83516 9.21051C5.83516 10.7323 6.87292 12.0169 8.28656 12.3769V14.1196Z" />,
text: "Cases",
description: "Case management"
},
{
image:
<path d="M6.93767 0C8.71083 0 10.4114 0.704386 11.6652 1.9582C12.919 3.21202 13.6234 4.91255 13.6234 6.68571C13.6234 8.34171 13.0165 9.864 12.0188 11.0366L12.2965 11.3143H13.1091L18.252 16.4571L16.7091 18L11.5662 12.8571V12.0446L11.2885 11.7669C10.116 12.7646 8.59367 13.3714 6.93767 13.3714C5.16451 13.3714 3.46397 12.667 2.21015 11.4132C0.956339 10.1594 0.251953 8.45888 0.251953 6.68571C0.251953 4.91255 0.956339 3.21202 2.21015 1.9582C3.46397 0.704386 5.16451 0 6.93767 0ZM6.93767 2.05714C4.36624 2.05714 2.3091 4.11429 2.3091 6.68571C2.3091 9.25714 4.36624 11.3143 6.93767 11.3143C9.5091 11.3143 11.5662 9.25714 11.5662 6.68571C11.5662 4.11429 9.5091 2.05714 6.93767 2.05714Z" />,
text: "SIEM",
description: "Case management"
},
{
image:
<path d="M11.223 10.971L3.85195 14.4L7.28095 7.029L14.652 3.6L11.223 10.971ZM9.25195 0C8.07006 0 6.89973 0.232792 5.8078 0.685084C4.71587 1.13738 3.72372 1.80031 2.88799 2.63604C1.20016 4.32387 0.251953 6.61305 0.251953 9C0.251953 11.3869 1.20016 13.6761 2.88799 15.364C3.72372 16.1997 4.71587 16.8626 5.8078 17.3149C6.89973 17.7672 8.07006 18 9.25195 18C11.6389 18 13.9281 17.0518 15.6159 15.364C17.3037 13.6761 18.252 11.3869 18.252 9C18.252 7.8181 18.0192 6.64778 17.5669 5.55585C17.1146 4.46392 16.4516 3.47177 15.6159 2.63604C14.7802 1.80031 13.788 1.13738 12.6961 0.685084C11.6042 0.232792 10.4338 0 9.25195 0ZM9.25195 8.01C8.98939 8.01 8.73758 8.1143 8.55192 8.29996C8.36626 8.48563 8.26195 8.73744 8.26195 9C8.26195 9.26256 8.36626 9.51437 8.55192 9.70004C8.73758 9.8857 8.98939 9.99 9.25195 9.99C9.51452 9.99 9.76633 9.8857 9.95199 9.70004C10.1376 9.51437 10.242 9.26256 10.242 9C10.242 8.73744 10.1376 8.48563 9.95199 8.29996C9.76633 8.1143 9.51452 8.01 9.25195 8.01Z" />,
text: "Assets",
description: "Case management"
},
{
image:
<path d="M13.3318 2.223C13.2598 2.223 13.1878 2.205 13.1248 2.169C11.3968 1.278 9.90284 0.9 8.11184 0.9C6.32984 0.9 4.63784 1.323 3.09884 2.169C2.88284 2.286 2.61284 2.205 2.48684 1.989C2.36984 1.773 2.45084 1.494 2.66684 1.377C4.34084 0.468 6.17684 0 8.11184 0C10.0288 0 11.7028 0.423 13.5388 1.368C13.7638 1.485 13.8448 1.755 13.7278 1.971C13.6468 2.133 13.4938 2.223 13.3318 2.223ZM0.452843 6.948C0.362843 6.948 0.272843 6.921 0.191843 6.867C-0.015157 6.723 -0.0601571 6.444 0.0838429 6.237C0.974843 4.977 2.10884 3.987 3.45884 3.294C6.28484 1.836 9.90284 1.827 12.7378 3.285C14.0878 3.978 15.2218 4.959 16.1128 6.21C16.2568 6.408 16.2118 6.696 16.0048 6.84C15.7978 6.984 15.5188 6.939 15.3748 6.732C14.5648 5.598 13.5388 4.707 12.3238 4.086C9.74084 2.763 6.43784 2.763 3.86384 4.095C2.63984 4.725 1.61384 5.625 0.803843 6.759C0.731843 6.885 0.596843 6.948 0.452843 6.948ZM6.07784 17.811C5.96084 17.811 5.84384 17.766 5.76284 17.676C4.97984 16.893 4.55684 16.389 3.95384 15.3C3.33284 14.193 3.00884 12.843 3.00884 11.394C3.00884 8.721 5.29484 6.543 8.10284 6.543C10.9108 6.543 13.1968 8.721 13.1968 11.394C13.1968 11.646 12.9988 11.844 12.7468 11.844C12.4948 11.844 12.2968 11.646 12.2968 11.394C12.2968 9.216 10.4158 7.443 8.10284 7.443C5.78984 7.443 3.90884 9.216 3.90884 11.394C3.90884 12.69 4.19684 13.887 4.74584 14.859C5.32184 15.894 5.71784 16.335 6.41084 17.037C6.58184 17.217 6.58184 17.496 6.41084 17.676C6.31184 17.766 6.19484 17.811 6.07784 17.811ZM12.5308 16.146C11.4598 16.146 10.5148 15.876 9.74084 15.345C8.39984 14.436 7.59884 12.96 7.59884 11.394C7.59884 11.142 7.79684 10.944 8.04884 10.944C8.30084 10.944 8.49884 11.142 8.49884 11.394C8.49884 12.663 9.14684 13.86 10.2448 14.598C10.8838 15.03 11.6308 15.237 12.5308 15.237C12.7468 15.237 13.1068 15.21 13.4668 15.147C13.7098 15.102 13.9438 15.264 13.9888 15.516C14.0338 15.759 13.8718 15.993 13.6198 16.038C13.1068 16.137 12.6568 16.146 12.5308 16.146ZM10.7218 18C10.6858 18 10.6408 17.991 10.6048 17.982C9.17384 17.586 8.23784 17.055 7.25684 16.092C5.99684 14.841 5.30384 13.176 5.30384 11.394C5.30384 9.936 6.54584 8.748 8.07584 8.748C9.60584 8.748 10.8478 9.936 10.8478 11.394C10.8478 12.357 11.6848 13.14 12.7198 13.14C13.7548 13.14 14.5918 12.357 14.5918 11.394C14.5918 8.001 11.6668 5.247 8.06684 5.247C5.51084 5.247 3.17084 6.669 2.11784 8.874C1.76684 9.603 1.58684 10.458 1.58684 11.394C1.58684 12.096 1.64984 13.203 2.18984 14.643C2.27984 14.877 2.16284 15.138 1.92884 15.219C1.69484 15.309 1.43384 15.183 1.35284 14.958C0.911843 13.779 0.695843 12.609 0.695843 11.394C0.695843 10.314 0.902843 9.333 1.30784 8.478C2.50484 5.967 5.15984 4.338 8.06684 4.338C12.1618 4.338 15.4918 7.497 15.4918 11.385C15.4918 12.843 14.2498 14.031 12.7198 14.031C11.1898 14.031 9.94784 12.843 9.94784 11.385C9.94784 10.422 9.11084 9.639 8.07584 9.639C7.04084 9.639 6.20384 10.422 6.20384 11.385C6.20384 12.924 6.79784 14.364 7.88684 15.444C8.74184 16.29 9.56084 16.758 10.8298 17.109C11.0728 17.172 11.2078 17.424 11.1448 17.658C11.0998 17.865 10.9108 18 10.7218 18Z" />,
text: "IAM",
description: "Case management"
},
{
image: <path d="M16.1091 8.57143H14.8234V5.14286C14.8234 4.19143 14.052 3.42857 13.1091 3.42857H9.68052V2.14286C9.68052 1.57454 9.45476 1.02949 9.0529 0.627628C8.65103 0.225765 8.10599 0 7.53767 0C6.96935 0 6.4243 0.225765 6.02244 0.627628C5.62057 1.02949 5.39481 1.57454 5.39481 2.14286V3.42857H1.96624C1.51158 3.42857 1.07555 3.60918 0.754056 3.93067C0.432565 4.25216 0.251953 4.6882 0.251953 5.14286V8.4H1.53767C2.82338 8.4 3.85195 9.42857 3.85195 10.7143C3.85195 12 2.82338 13.0286 1.53767 13.0286H0.251953V16.2857C0.251953 16.7404 0.432565 17.1764 0.754056 17.4979C1.07555 17.8194 1.51158 18 1.96624 18H5.22338V16.7143C5.22338 15.4286 6.25195 14.4 7.53767 14.4C8.82338 14.4 9.85195 15.4286 9.85195 16.7143V18H13.1091C13.5638 18 13.9998 17.8194 14.3213 17.4979C14.6428 17.1764 14.8234 16.7404 14.8234 16.2857V12.8571H16.1091C16.6774 12.8571 17.2225 12.6314 17.6243 12.2295C18.0262 11.8277 18.252 11.2826 18.252 10.7143C18.252 10.146 18.0262 9.60092 17.6243 9.19906C17.2225 8.79719 16.6774 8.57143 16.1091 8.57143Z" />,
text: "Intel",
description: "Case management"
},
{
image:
<path d="M9.89516 7.71433H8.60945V5.1429H9.89516V7.71433ZM9.89516 10.2858H8.60945V9.00004H9.89516V10.2858ZM14.3952 2.57147H4.10944C3.76845 2.57147 3.44143 2.70693 3.20031 2.94805C2.95919 3.18917 2.82373 3.51619 2.82373 3.85719V15.4286L5.39516 12.8572H14.3952C14.7362 12.8572 15.0632 12.7217 15.3043 12.4806C15.5454 12.2395 15.6809 11.9125 15.6809 11.5715V3.85719C15.6809 3.14361 15.1023 2.57147 14.3952 2.57147Z" />,
text: "Comms",
description: "Case management"
},
{
image:
<path d="M0.251953 10.6011H3.8391L9.38052 -4.92572e-08L10.8977 11.5696L15.0377 6.28838L19.3191 10.6011H23.3948V13.1836H18.252L15.2562 10.175L9.1491 18L7.88909 8.41894L5.39481 13.1836H0.251953V10.6011Z" />,
text: "Network",
description: "Case management"
},
{
image:
<path d="M19.1722 8.9957L17.0737 6.60487L17.3661 3.44004L14.2615 2.73483L12.6361 -3.28068e-08L9.71206 1.25561L6.78803 -3.28068e-08L5.16261 2.73483L2.05797 3.43144L2.35038 6.59627L0.251953 8.9957L2.35038 11.3865L2.05797 14.56L5.16261 15.2652L6.78803 18L9.71206 16.7358L12.6361 17.9914L14.2615 15.2566L17.3661 14.5514L17.0737 11.3865L19.1722 8.9957ZM10.5721 13.2957H8.85205V11.5757H10.5721V13.2957ZM10.5721 9.85571H8.85205V4.69565H10.5721V9.85571Z" />,
text: "EDR & AV",
description: "Case management"
},
]
const LandingpageUsecases = (props) => {
const [selectedUsecase, setSelectedUsecase] = useState("Phishing")
const usecasekeys = usecases === undefined || usecases === null ? [] : Object.keys(usecases)
const buttonBackground = "linear-gradient(to right, #f86a3e, #f34079)"
const buttonStyle = {borderRadius: 25, height: 50, width: 260, margin: isMobile ? "15px auto 15px auto" : 20, fontSize: 18, backgroundImage: buttonBackground}
const HandleTitle = (props) => {
const { usecases, selectedUsecase, setSelecedUsecase } = props
const [progress, setProgress] = useState(0)
useEffect(() => {
const timer = setInterval(() => {
setProgress((oldProgress) => {
if (oldProgress >= 105) {
const foundIndex = usecasekeys.findIndex(key => key === selectedUsecase)
var newitem = usecasekeys[foundIndex+1]
if (newitem === undefined || newitem === 0) {
newitem = usecasekeys[1]
}
setSelectedUsecase(newitem)
return -18
}
if (oldProgress >= 65) {
return oldProgress + 3
}
if (oldProgress >= 80) {
return oldProgress + 1
}
return oldProgress + 6
})
}, 165)
return () => {
clearInterval(timer)
}
}, [])
if (usecases === null || usecases === undefined || usecases.length === 0) {
return null
}
const modifier = isMobile ? 17 : 22
return (
<span style={{margin: "auto", textAlign: isMobile ? "center" : "left", width: isMobile ? 280 : "100%",}}>
<b>Handle <br/>
<span style={{marginBottom: 10}}>
<i id="usecase-text">{selectedUsecase}</i>
<LinearProgress variant="determinate" value={progress} style={{marginTop: 0, marginBottom: 0, height: 3, width: isMobile ? "100%" : selectedUsecase.length*modifier, borderRadius: 10, }} />
</span>
with confidence</b>
</span>
)
}
const parsedWidth = isMobile ? "100%" : 1100
return (
<div style={{width: isMobile ? null : parsedWidth, margin: isMobile ? "0px 0px 0px 0px" : "auto", color: "white", textAlign: isMobile ? "center" : "left",}}>
<div style={{display: "flex", position: "relative",}}>
<div style={{maxWidth: isMobile ? "100%" : 420, paddingTop: isMobile ? 0 : 120, zIndex: 1000, margin: "auto",}}>
<Typography variant="h1" style={{margin: "auto", width: isMobile ? 280 : "100%", marginTop: isMobile ? 50 : 0}}>
<HandleTitle usecases={usecases} selectedUsecase={selectedUsecase} setSelectedUsecase={setSelectedUsecase} />
{/*<b>Security Automation <i>is Hard</i></b>*/}
</Typography>
<Typography variant="h6" style={{marginTop: isMobile ? 15 : 0,}}>
Connecting your everchanging environment is hard. We get it! That's why we built Shuffle, where you can use and share your security workflows to everyones benefit.
{/*Shuffle is an automation platform where you don't need to be an expert to automate. Get access to our large pool of security playbooks, apps and people.*/}
</Typography>
<div style={{display: "flex", textAlign: "center", itemAlign: "center",}}>
{isMobile ? null :
<Link rel="noopener noreferrer" to={"/pricing"} style={{textDecoration: "none"}}>
<Button
variant="contained"
onClick={() => {
ReactGA.event({
category: "landingpage",
action: "click_main_pricing",
label: "",
})
}}
style={{
borderRadius: 25, height: 40, width: 175, margin: "15px 0px 15px 0px", fontSize: 14, color: "white", backgroundImage: buttonBackground, marginRight: 10,
}}>
See Pricing
</Button>
</Link>
}
{isMobile ? null :
<Link rel="noopener noreferrer" to={"/register?message=You'll need to sign up first. No name, company or credit card required."} style={{textDecoration: "none"}}>
<Button
variant="contained"
onClick={() => {
ReactGA.event({
category: "landingpage",
action: "click_main_try_it_out",
label: "",
})
}}
style={{
borderRadius: 25, height: 40, width: 175, margin: "15px 0px 15px 0px", fontSize: 14, color: "white", backgroundImage: buttonBackground,
}}>
Start for free
</Button>
</Link>
}
</div>
</div>
{isMobile ? null :
<div style={{marginLeft: 200, marginTop: 125, zIndex: 1000}}>
<DetectionFramework showOptions={false} selectedOption={selectedUsecase} rolling={true} />
</div>
}
{isMobile ? null :
<div style={{position: "absolute", top: 50, right: -200, zIndex: 0, }}>
<svg width="351" height="433" viewBox="0 0 351 433" fill="none" xmlns="http://www.w3.org/2000/svg" style={{zIndex: 0, }}>
<path d="M167.781 184.839C167.781 235.244 208.625 276.104 259.03 276.104C309.421 276.104 350.28 235.244 350.28 184.839C350.28 134.448 309.421 93.5892 259.03 93.5892C208.625 93.5741 167.781 134.433 167.781 184.839ZM330.387 184.839C330.387 224.263 298.439 256.195 259.03 256.195C219.621 256.195 187.674 224.248 187.674 184.839C187.674 145.43 219.636 113.483 259.03 113.483C298.439 113.483 330.387 145.43 330.387 184.839Z" fill="white" fill-opacity="0.2"/>
<path d="M167.781 387.368C167.781 412.578 188.203 433 213.398 433C238.593 433 259.03 412.578 259.03 387.368C259.03 362.157 238.608 341.735 213.398 341.735C188.187 341.735 167.781 362.172 167.781 387.368ZM249.076 387.368C249.076 407.08 233.095 423.046 213.398 423.046C193.686 423.046 177.72 407.065 177.72 387.368C177.72 367.671 193.686 351.69 213.398 351.69C233.095 351.705 249.076 367.671 249.076 387.368Z" fill="white" fill-opacity="0.2"/>
<path d="M56.8637 0.738726C25.7052 0.738724 0.44632 25.9976 0.446317 57.1561C0.446314 88.3146 25.7052 113.573 56.8637 113.573C88.0221 113.573 113.281 88.3146 113.281 57.1561C113.281 25.9977 88.0222 0.738729 56.8637 0.738726Z" fill="white" fill-opacity="0.2"/>
</svg>
</div>
}
</div>
<div style={{display: "flex", width: isMobile ? "100%" : 300, itemAlign: "center", margin: "auto", marginTop: 20, flexDirection: isMobile ? "column" : "row", textAlign: "center",}}>
{isMobile ?
<Link rel="noopener noreferrer" to={"/pricing"} style={{textDecoration: "none"}}>
<Button
variant={isMobile ? "contained" : "outlined"}
color={isMobile ? "primary" : "secondary"}
style={buttonStyle}
onClick={() => {
ReactGA.event({
category: "landingpage",
action: "click_main_pricing",
label: "",
})
}}
>
See pricing
</Button>
</Link>
: null
}
{/*isMobile ?
<Link rel="noopener noreferrer" to={"/docs/features"} style={{textDecoration: "none"}}>
<Button
variant="outlined"
onClick={() => {
ReactGA.event({
category: "landingpage",
action: "click_main_features",
label: "",
})
}}
color="secondary"
style={buttonStyle}>
Features
</Button>
</Link>
: null*/}
</div>
{isMobile ? null :
<div style={{display: "flex", width: parsedWidth, margin: "auto", marginTop: 150}}>
{securityFramework.map((data, index) => {
return (
<div key={index} style={{flex: 1, textAlign: "center",}}>
<span style={{margin: "auto", width: 25,}}>
<svg width="25" height="25" fill="white" xmlns="http://www.w3.org/2000/svg" >
{data.image}
</svg>
</span>
<Typography variant="body2" style={{color: "white", marginRight: 5}}>
{data.text}
</Typography>
</div>
)
})}
</div>
}
</div>
)
}
export default LandingpageUsecases;
frontend/src/components/LoginPopup.js
+160 -123
View File
@@ -1,139 +1,176 @@
/* eslint-disable react/no-multi-comp */
import React, {useState} from 'react';
import React, { useState } from "react";
import DialogTitle from '@material-ui/core/DialogTitle';
import Dialog from '@material-ui/core/Dialog';
import TextField from '@material-ui/core/TextField';
import Button from '@material-ui/core/Button';
import DialogTitle from "@material-ui/core/DialogTitle";
import Dialog from "@material-ui/core/Dialog";
import TextField from "@material-ui/core/TextField";
import Button from "@material-ui/core/Button";
const LoginDialog = props => {
const { classes, onClose, open, globalUrl, isLoggedIn, setIsLoggedIn, ...other } = props;
const LoginDialog = (props) => {
const {
classes,
onClose,
open,
globalUrl,
isLoggedIn,
setIsLoggedIn,
...other
} = props;
const [username, setUsername] = useState("");
const [password, setPassword] = useState("");
//const [selectedValue, setSelectedValue] = useState(false);
const [username, setUsername] = useState("");
const [password, setPassword] = useState("");
//const [selectedValue, setSelectedValue] = useState(false);
// Used to swap from login to register. True = login, false = register
const [loginCheck, setLoginCheck] = useState(true);
// Used to swap from login to register. True = login, false = register
const [loginCheck, setLoginCheck] = useState(true);
// Error messages etc
const [loginInfo, setLoginInfo] = useState("");
// Error messages etc
const [loginInfo, setLoginInfo] = useState("");
const handleValidateForm = () => {
return (username.length > 1 && password.length > 8);
}
const handleValidateForm = () => {
return username.length > 1 && password.length > 8;
};
const onSubmit = (e) => {
e.preventDefault()
const onSubmit = (e) => {
e.preventDefault();
// Just use this one?
var data = '{"username": "' + username + '", "password": "' + password + '"}';
var baseurl = globalUrl
if (loginCheck) {
var url = baseurl+'/login';
fetch(url, {
method: 'POST',
body: data,
headers: {
'Content-Type': 'application/json',
},
})
.then(response =>
response.json().then(responseJson => {
console.log(responseJson)
//console.log(e)
if (responseJson["success"] === false) {
setLoginInfo(responseJson["reason"])
} else {
setLoginInfo("Successful login :)")
onClose()
setIsLoggedIn(true)
}
}),
)
.catch(error => {
setLoginInfo("Error in userdata")
});
} else {
url = baseurl+'/register';
fetch(url, {
method: 'POST',
body: data,
headers: {
'Content-Type': 'application/json',
},
})
.then(response =>
response.json().then(responseJson => {
if (responseJson["success"] === false) {
setLoginInfo(responseJson["reason"])
} else {
setLoginInfo("Successful register :)")
onClose()
setIsLoggedIn(true)
}
}),
)
.catch(error => {
setLoginInfo("Error in userdata")
});
}
}
// Just use this one?
var data =
'{"username": "' + username + '", "password": "' + password + '"}';
var baseurl = globalUrl;
if (loginCheck) {
var url = baseurl + "/login";
fetch(url, {
method: "POST",
body: data,
headers: {
"Content-Type": "application/json",
},
})
.then((response) =>
response.json().then((responseJson) => {
console.log(responseJson);
//console.log(e)
if (responseJson["success"] === false) {
setLoginInfo(responseJson["reason"]);
} else {
setLoginInfo("Successful login :)");
onClose();
setIsLoggedIn(true);
}
})
)
.catch((error) => {
setLoginInfo("Error in userdata");
});
} else {
url = baseurl + "/register";
fetch(url, {
method: "POST",
body: data,
headers: {
"Content-Type": "application/json",
},
})
.then((response) =>
response.json().then((responseJson) => {
if (responseJson["success"] === false) {
setLoginInfo(responseJson["reason"]);
} else {
setLoginInfo("Successful register :)");
onClose();
setIsLoggedIn(true);
}
})
)
.catch((error) => {
setLoginInfo("Error in userdata");
});
}
};
const onChangeUser = (e) => {
setUsername(e.target.value)
}
const onChangeUser = (e) => {
setUsername(e.target.value);
};
const onChangePass = (e) => {
setPassword(e.target.value)
}
const onChangePass = (e) => {
setPassword(e.target.value);
};
const onClickRegister = () => {
setLoginCheck(!loginCheck)
}
const onClickRegister = () => {
setLoginCheck(!loginCheck);
};
//var loginChange = loginCheck ? (<div><p onClick={setLoginCheck(false)}>Want to register? Click here.</p></div>) : (<div><p onClick={setLoginCheck(true)}>Go back to login? Click here.</p></div>);
var formtitle = loginCheck ? <div>Login</div> : <div>Register</div>
var formButton = loginCheck ? <div>Click to Register</div> : <div>Click to Login</div>
//var loginChange = loginCheck ? (<div><p onClick={setLoginCheck(false)}>Want to register? Click here.</p></div>) : (<div><p onClick={setLoginCheck(true)}>Go back to login? Click here.</p></div>);
var formtitle = loginCheck ? <div>Login</div> : <div>Register</div>;
var formButton = loginCheck ? (
<div>Click to Register</div>
) : (
<div>Click to Login</div>
);
return (
<Dialog modal open={open} onClose={onClose} {...other}>
<DialogTitle>{formtitle}</DialogTitle>
<form onSubmit={onSubmit} style={{margin: "15px 15px 15px 15px"}}>
Username
<div>
<TextField
required
id="standard-required"
autoComplete="username"
margin="normal"
variant="outlined"
onChange={onChangeUser}
/>
</div>
Password
<div>
<TextField
id="outlined-password-input"
type="password"
autoComplete="current-password"
margin="normal"
variant="outlined"
onChange={onChangePass}
/>
</div>
<div style={{display: "flex", marginTop: "15px"}}>
<Button color="secondary" variant="contained" type="submit" style={{flex: "1", marginRight: "5px"}} disabled={!handleValidateForm()}>SUBMIT</Button>
<Button color="primary" variant="contained" type="button" style={{flex: "1"}} onClick={onClose}>Cancel</Button>
</div>
{loginInfo}
</form>
<div style={{display: "flex"}}>
<Button color="secondary" variant="contained" onClick={onClickRegister} type="button" style={{flex: "1"}}>{formButton}</Button>
</div>
</Dialog>
);
}
return (
<Dialog modal open={open} onClose={onClose} {...other}>
<DialogTitle>{formtitle}</DialogTitle>
<form onSubmit={onSubmit} style={{ margin: "15px 15px 15px 15px" }}>
Username
<div>
<TextField
required
id="standard-required"
autoComplete="username"
margin="normal"
variant="outlined"
onChange={onChangeUser}
/>
</div>
Password
<div>
<TextField
id="outlined-password-input"
type="password"
autoComplete="current-password"
margin="normal"
variant="outlined"
onChange={onChangePass}
/>
</div>
<div style={{ display: "flex", marginTop: "15px" }}>
<Button
color="secondary"
variant="contained"
type="submit"
style={{ flex: "1", marginRight: "5px" }}
disabled={!handleValidateForm()}
>
SUBMIT
</Button>
<Button
color="primary"
variant="contained"
type="button"
style={{ flex: "1" }}
onClick={onClose}
>
Cancel
</Button>
</div>
{loginInfo}
</form>
<div style={{ display: "flex" }}>
<Button
color="secondary"
variant="contained"
onClick={onClickRegister}
type="button"
style={{ flex: "1" }}
>
{formButton}
</Button>
</div>
</Dialog>
);
};
export default LoginDialog;

Some files were not shown because too many files have changed in this diff Show More