Merge pull request #1 from dhaval055/patch-1

Patch 1

.dockerignore

@@ -1,15 +0,0 @@
-**/.git*
-**/.gitlab-ci.yml
-**/.env
-**/.dockerignore
-**/Dockerfile*
-**/node_modules
-buildSrc/libs
-.gradle/
-build/
-**build/
-.idea/
-!.idea/codeStyles/codeStyleConfig.xml
-.DS_Store
-*.log
-out/

.env

@@ -2,16 +2,18 @@
 ORG_ID=Shuffle
 ENVIRONMENT_NAME=Shuffle
 
+
 # Remote github config for first load
 SHUFFLE_DOWNLOAD_WORKFLOW_LOCATION=
 SHUFFLE_DOWNLOAD_WORKFLOW_USERNAME=
 SHUFFLE_DOWNLOAD_WORKFLOW_PASSWORD=
 SHUFFLE_DOWNLOAD_WORKFLOW_BRANCH=
 
-SHUFFLE_APP_DOWNLOAD_LOCATION=https://github.com/frikky/shuffle-apps 
+SHUFFLE_APP_DOWNLOAD_LOCATION=https://github.com/shuffle/python-apps
 SHUFFLE_DOWNLOAD_AUTH_USERNAME=
 SHUFFLE_DOWNLOAD_AUTH_PASSWORD=
 SHUFFLE_DOWNLOAD_AUTH_BRANCH=
+SHUFFLE_APP_FORCE_UPDATE=false
 
 # User config for first load. Username & PW: min length 3
 SHUFFLE_DEFAULT_USERNAME=
@@ -20,26 +22,55 @@ SHUFFLE_DEFAULT_APIKEY=
 
 # Local location of your app directory. Can't use ~/
 # Files will get better at some point. Right now: local saving.
+SHUFFLE_APP_HOTLOAD_FOLDER=./shuffle-apps
 SHUFFLE_APP_HOTLOAD_LOCATION=./shuffle-apps
 SHUFFLE_FILE_LOCATION=./shuffle-files
 
+# Encryption modifier. This HAS to be set to encrypt any authentication being used in Shuffle. This is put together with other relevant values to ensure multiple parts are needed to decrypt. 
+# If this key is lost or changed, you will have to reauthenticate all apps.
+SHUFFLE_ENCRYPTION_MODIFIER=
+
 # Other configs
+BASE_URL=http://shuffle-backend:5001 
 BACKEND_HOSTNAME=shuffle-backend
 BACKEND_PORT=5001
 FRONTEND_PORT=3001
 FRONTEND_PORT_HTTPS=3443
+
+# CHANGE THIS IF YOU WANT GOOD LOCAL EXECUTIONS:
 OUTER_HOSTNAME=shuffle-backend
 DB_LOCATION=./shuffle-database
+DOCKER_API_VERSION=1.40
 
 # Proxy configurations. SHUFFLE_PASS_WORKER_PROXY must be FALSE to not pass the proxy information to sub-apps.
 # PS: It will skip proxy for 
-SHUFFLE_HTTP_PROXY=
+HTTP_PROXY=
-SHUFFLE_HTTPS_PROXY=
+HTTPS_PROXY=
 SHUFFLE_PASS_WORKER_PROXY=TRUE
+SHUFFLE_PASS_APP_PROXY=FALSE
+TZ=Europe/Amsterdam 					# Timezone-handler in Orborus, Worker and Apps
+ORBORUS_CONTAINER_NAME=				# Used to FIND the containername. cgroup v2: issue 501
 
-SHUFFLE_BASE_IMAGE_REGISTRY=ghcr.io
 SHUFFLE_BASE_IMAGE_NAME=frikky
-SHUFFLE_BASE_IMAGE_TAG_SUFFIX="-0.8.3"
+SHUFFLE_BASE_IMAGE_REGISTRY=ghcr.io
+SHUFFLE_BASE_IMAGE_TAG_SUFFIX="-0.8.80"
 
 # Used for auto-cleanup of containers. REALLY important at scale.
 SHUFFLE_CONTAINER_AUTO_CLEANUP=false
+SHUFFLE_ELASTIC=true
+SHUFFLE_LOGS_DISABLED=false
+SHUFFLE_CHAT_DISABLED=false
+SHUFFLE_RERUN_SCHEDULE=300
+
+# DATABASE CONFIGURATIONS
+DATASTORE_EMULATOR_HOST=shuffle-database:8000
+#SHUFFLE_OPENSEARCH_URL=http://shuffle-opensearch:9200
+SHUFFLE_OPENSEARCH_URL=https://shuffle-opensearch:9200
+SHUFFLE_OPENSEARCH_USERNAME=admin
+SHUFFLE_OPENSEARCH_PASSWORD=admin
+SHUFFLE_OPENSEARCH_CERTIFICATE_FILE=
+SHUFFLE_OPENSEARCH_APIKEY=
+SHUFFLE_OPENSEARCH_CLOUDID=
+SHUFFLE_OPENSEARCH_PROXY=
+SHUFFLE_OPENSEARCH_INDEX_PREFIX=
+SHUFFLE_OPENSEARCH_SKIPSSL_VERIFY=true

.github/CONTRIBUTING.md

@@ -0,0 +1,48 @@
+# Contributing to Shuffle
+
+First off, thank you for contributing to [Shuffle](https://shuffler.io/)! Your talents and your contributions are greatly appreciated. With Shuffle, we aim to make cybersecurity more accessible, and keep that in mind with everything we make. 
+
+## Opening a new issue
+
+If you find a bug or think of an improvement or fix, please open a [new issue](https://github.com/frikky/Shuffle/issues/new). Outline every step necessary to reproduce the bug. Include screenshots, logs and/or code examples where applicable. The more thorough you are, the better.
+
+## What you can work on
+There are a lot of things to work on in our complex ecosystem. The most pressing issues are documentation, use-cases and content-creation, but any help is appreciated. We'll make sure you get the help you need to get started. Below is an incomplete list of items. If you see an issue, tell us or fix it! :)
+
+#### App Creation (Python & GUI w/OpenAPI)
+As with everything else, app creation for Shuffle is made as accessibl as possible with the app editor. However, there are some instances where it can't do the job, and you'll have to write Python code. The App Editor generates OpenAPI specifications and can be widely shared, while Python apps only work for Shuffle and NSA's WALKOFF (which Shuffle is based on). You can find our [OpenAPI apps here](https://github.com/frikky/security-openapis) and our [Python apps here](https://github.com/frikky/shuffle-apps). Apps in these repositories are automatically available after installation. Shuffle apps are searchable on [https://shuffler.io](https://shuffler.io/search).
+
+#### Workflow creation (GUI & Conceptualizing) 
+Workflows are where the magic of Shuffle automation happens. Our current ones [are outlined here](https://github.com/frikky/security-openapis), and will be automatically imported into Shuffle instances in the future. They are split into Prepare and Response, but don't necessarily have to be. If you'd like to talk about workflow creation or use-cases in general, either Open a [new issue](https://github.com/frikky/shuffle-workflows/issues/new) or send us an email at [frikky@shuffler.io](mailto:frikky@shuffler.io) 
+
+#### Documentation (Markdown)
+Documentation is essential to any product, and Shuffle is no exception. Documentation in Shuffle uses markdown and is located in the [shuffle-docs](https://github.com/frikky/shuffle-docs/tree/master/docs) repository. These are then loaded into Shuffle when someone visits [https://shuffler/docs/about](https://shuffler/docs/about), then cached for later use. If you make an edit, expect it on our website in about an hour. 
+
+#### Frontend (ReactJS)
+The frontend of Shuffle is what everyone sees when they log in. Our goal here is to make it easy to get started and keep going with Shuffle - removing any blockers from the point of accessibility. If you'd like to get started, find [an issue](https://github.com/frikky/Shuffle/issues) and check the [installation guide](https://github.com/frikky/Shuffle/blob/master/install-guide.md#local-development-installation) for setting it up locally without Docker. 
+
+#### Backend (Golang)
+The backend of Shuffle is our REST API Server that runs in the background, handling all the API-calls in general, whether from users or apps. If you'd like to get started, find [an issue](https://github.com/frikky/Shuffle/issues) and check the [installation guide](https://github.com/frikky/Shuffle/blob/master/install-guide.md#local-development-installation) for setting it up locally without Docker. 
+
+#### Scaling (Golang & Python)
+Shuffle runs using Docker, and is built to scale. There are many areas that may revolve around scaling, but the main issues come down to how we use Docker in our [architecture](https://shuffler.io/docs/architecture). If you want to help by submitting Helm charts (K8s), Docker swarm configurations, blogposts, or talk about code changes that would help scaling - please reach out (or just start building!), and we can discuss the possibilities. Make sure to read about the architecture first :)
+
+#### Testing 
+Whether it's security testing, code testing or CI/CD, we could always need another hand. E.g. an example of CI/CD used for apps can be found [here](https://github.com/Shuffle/Shuffle-apps/blob/master/.github/workflows/ci.yaml), but we don't at all limit the scope to Github actions. If you find a security issue, whether open source or not, please contact [security@shuffler.io](mailto:security@shuffler.io) or [contact us on our website](https://shuffler.io/contact).
+
+#### Community 
+What is a product without a community? Want to help out? Whether it be through blogposts, videos or community management, don't hesitate to [reach out](https://shuffler.io/contact) if you would like to help, and get a more keen understanding of how we work. (PS: We're hiring)
+
+## Working on an issue
+
+**Shuffle** uses the [GitHub flow](https://guides.github.com/introduction/flow/index.html). All project changes are made through pull requests.
+If you see an issue that you would like to work on, leave a quick comment or just get cracking. 
+
+### License
+
+All contributions are made under either the **GNU Affero General Public License v3.0** or **MIT** license. See below for further details.
+
+* [Main project license - AGPLv3](https://github.com/frikky/Shuffle/blob/master/LICENSE)
+* [Apps - MIT](https://github.com/frikky/Shuffle-apps/blob/master/LICENSE)
+* [Workflows - MIT](https://github.com/frikky/Shuffle-workflows/blob/master/LICENSE)
+* [Documentation - MIT](https://github.com/frikky/Shuffle-docs/blob/master/LICENSE)

.github/install-guide.md

@@ -0,0 +1,127 @@
+# Installation guide
+Installation of Shuffle is currently only available in docker. Looking for how to update Shuffle? Check the [updating guide](https://shuffler.io/docs/configuration#updating_shuffle)
+
+This document outlines a an introduction environment which is not scalable. [Read here](https://shuffler.io/docs/configuration#production_readiness) for information on production readiness. This also includes system requirements and configurations for Swarm or K8s. 
+
+# Docker - *nix
+The Docker setup is done with docker-compose 
+
+**PS: if you're setting up Shuffle on Windows, go to the next step (Windows Docker setup)**
+
+1. Make sure you have [Docker](https://docs.docker.com/get-docker/) and [docker-compose](https://docs.docker.com/compose/install/) installed.
+2. Download Shuffle
+```bash
+git clone https://github.com/frikky/Shuffle
+cd Shuffle
+```
+
+3. Fix prerequisites for the Opensearch database (Elasticsearch): 
+```bash
+mkdir shuffle-database
+sudo chown -R 1000:1000 shuffle-database
+```
+
+4. Run docker-compose.
+```bash
+docker-compose up -d
+```
+
+When you're done, skip to the "After installation" step below.
+
+## Windows with WSL  
+This step is for setting up with Docker on windows from scratch.
+
+1. Make sure you have [Docker](https://docs.docker.com/docker-for-windows/install/) and [docker-compose](https://docs.docker.com/compose/install/) installed. WSL2 may be required.
+
+2. Go to https://github.com/frikky/shuffle/releases and download the latest .zip release (or install git)
+
+3. Unzip the folder and enter it
+
+4. Open the .env file and change the line with "OUTER_HOSTNAME" to contain your IP:
+
+```bash
+OUTER_HOSTNAME=YOUR.IP.HERE
+```
+
+6. Run docker-compose
+```bash
+docker-compose up -d
+```
+
+### Configurations (proxies, default users etc.)
+https://shuffler.io/docs/configuration
+
+### After installation 
+1. After installation, go to http://localhost:3001 (or your servername - https is on port 3443)
+2. Now set up your admin account (username & password). Shuffle doesn't have a default username and password. 
+3. Sign in with the same Username & Password! Go to /apps and see if you have any apps yet. If not - you may need to [configure proxies](https://shuffler.io/docs/configuration#production_readiness)
+4. Check out https://shuffler.io/docs/configuration as it has a lot of useful information to get started
+
+![Admin account setup](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_adminaccount.png)
+
+### Useful info
+* Check out [getting started](https://shuffler.io/docs/getting_started)
+* The default state of Shuffle is NOT scalable. See [production setup](https://shuffler.io/docs/configuration#production_readiness) for more info
+* The server is available on http://localhost:3001 (or your servername)
+* Further configurations can be done in docker-compose.yml and .env.
+* Default database location is in the same folder: ./shuffle-database
+
+# Local development installation
+
+Local development is pretty straight forward with **ReactJS** and **Golang**. This part is intended to help you run the code for development purposes. We recommend having Shuffle running with the Docker-compose, then manually running the portion that you want to test and/or edit.
+
+**PS: You have to stop the Backend Docker container to get this one working**
+
+**PPS: Use the "Launch" branch when developing to get it set up easier**
+
+## Frontend - ReactJS /w cytoscape
+http://localhost:3000 - Requires [npm](https://nodejs.org/en/download/)/[yarn](https://yarnpkg.com/lang/en/docs/install/#debian-stable)/your preferred manager. Runs independently from backend.
+```bash
+cd frontend
+npm i
+npm start
+```
+
+## Backend - Golang
+http://localhost:5001 - REST API - requires [>=go1.13](https://golang.org/dl/)
+```bash
+export SHUFFLE_OPENSEARCH_URL="http://localhost:9200"
+export SHUFFLE_ELASTIC=true
+cd backend/go-app
+go run main.go walkoff.go docker.go
+```
+
+Large portions of the backend is written in another repository - [shuffle-shared](https://github.com/frikky/shuffle-shared). If you want to update any of this code and test in realtime, we recommend following these steps:
+1. Clone shuffle-shared to a local repository
+2. Open the Shuffle backend's go.mod file (./shuffle/backend/go.mod)  (**NOT** in shuffle-shared)
+3. Change the following line to point to your directory AFTER the =>
+```
+//replace github.com/frikky/shuffle-shared => ../../../../git/shuffle-shared
+```
+4. Make the changes you want, then restart the backend server!
+5. With your changes made, make a pull request :fire:
+
+**WINDOWS USERS:** You'll have to to add the "export" part as an environment variable.
+
+## Database - Opensearch 
+Make sure this is running through the docker-compose, and that the backend points to it with SHUFFLE_OPENSEARCH_URL defined
+
+## Orborus
+Execution of Workflows:
+PS: This requires some specific environment variables
+```
+cd functions/onprem/orborus
+go run orborus.go
+```
+
+Environments (modify for Windows):
+```
+export ORG_ID=Shuffle
+export ENVIRONMENT_NAME=Shuffle
+export BASE_URL=http://YOUR-IP:5001
+export DOCKER_API_VERSION=1.40
+```
+
+**WINDOWS USERS:** You'll have to to add the "export" part as an environment variable.
+
+AND THAT's it - hopefully it worked. If it didn't please email [frikky@shuffler.io](mailto:frikky@shuffler.io)

.github/workflows/ci.yaml

@@ -0,0 +1,66 @@
+name: ci
+
+on:
+  push:
+    branches: master
+jobs:
+  main:
+    runs-on: ubuntu-latest
+    continue-on-error: ${{ matrix.experimental }}
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+         - app: frontend
+           path: frontend
+           version: 0.8.3
+           experimental: true
+         - app: backend
+           path: backend
+           version: 0.8.3
+           experimental: false
+         - app: orborus
+           path: functions/onprem/orborus
+           version: 0.8.0
+           experimental: false
+         - app: database
+           path: backend/database
+           version: 0.8.0
+           experimental: false
+    steps:
+      -
+        name: Checkout
+        uses: actions/checkout@v2
+      -
+        name: Set up QEMU
+        uses: docker/setup-qemu-action@v1
+      -
+        name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v1
+      -
+        name: Login to DockerHub
+        uses: docker/login-action@v1 
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+#        Use below configuration for ghcr.io
+#        with:
+#          registry: ghcr.io
+#          username: ${{ github.repository_owner }}
+#          password: ${{ secrets.CR_PAT }} 
+      -
+        name: Build and push
+        id: docker_build
+        uses: docker/build-push-action@v2
+        env:
+          BUILDX_NO_DEFAULT_LOAD: true
+        with:
+          context: ${{ matrix.path }}/
+          file: ${{ matrix.path }}/Dockerfile          
+          platforms: linux/amd64,linux/arm64
+          #,linux/386 - no node image I guess?
+          push: true
+          tags: ${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.app }}:${{ matrix.version }}
+      -
+        name: Image digest
+        run: echo ${{ steps.docker_build.outputs.digest }}

.github/workflows/codeql-analysis.yml

@@ -13,10 +13,14 @@ name: "CodeQL"
 
 on:
   push:
-    branches: [ master ]
+    branches: 
+      - master
+      - launch 
   pull_request:
     # The branches below must be a subset of the branches above
-    branches: [ master ]
+    branches: 
+      - master
+      - launch 
   schedule:
     - cron: '38 16 * * 4'
 

.github/workflows/snyk-container-analysis.yml

@@ -0,0 +1,47 @@
+# A sample workflow which checks out the code, builds a container
+# image using Docker and scans that image for vulnerabilities using
+# Snyk. The results are then uploaded to GitHub Security Code Scanning
+#
+# For more examples, including how to limit scans to only high-severity
+# issues, monitor images for newly disclosed vulnerabilities in Snyk and
+# fail PR checks for new vulnerabilities, see https://github.com/snyk/actions/
+
+name: Snyk Container
+
+on:
+  push:
+    branches: 
+      - master
+      - launch 
+  pull_request:
+    # The branches below must be a subset of the branches above
+    branches: 
+      - master
+      - launch 
+  schedule:
+    - cron: '18 4 * * 3'
+
+jobs:
+  snyk:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: Build a Docker image
+      run: docker build -t your/image-to-test .
+    - name: Run Snyk to check Docker image for vulnerabilities
+      # Snyk can be used to break the build when it detects vulnerabilities.
+      # In this case we want to upload the issues to GitHub Code Scanning
+      continue-on-error: true
+      uses: snyk/actions/docker@master
+      env:
+        # In order to use the Snyk Action you will need to have a Snyk API token.
+        # More details in https://github.com/snyk/actions#getting-your-snyk-token
+        # or you can signup for free at https://snyk.io/login
+        SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
+      with:
+        image: your/image-to-test
+        args: --file=Dockerfile
+    - name: Upload result to GitHub Code Scanning
+      uses: github/codeql-action/upload-sarif@v1
+      with:
+        sarif_file: snyk.sarif

.github/workflows/snyk-infrastructure-analysis.yml

@@ -0,0 +1,46 @@
+# A sample workflow which checks out your Infrastructure as Code Configuration files,
+# such as Kubernetes, Helm & Terraform and scans them for any security issues.
+# The results are then uploaded to GitHub Security Code Scanning
+#
+# For more examples, including how to limit scans to only high-severity issues
+# and fail PR checks, see https://github.com/snyk/actions/
+
+name: Snyk Infrastructure as Code
+
+on:
+  push:
+    branches: 
+      - master
+      - launch 
+  pull_request:
+    # The branches below must be a subset of the branches above
+    branches:
+      - master
+      - launch 
+  schedule:
+    - cron: '41 16 * * 2'
+
+jobs:
+  snyk:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Run Snyk to check configuration files for security issues
+        # Snyk can be used to break the build when it detects security issues.
+        # In this case we want to upload the issues to GitHub Code Scanning
+        continue-on-error: true
+        uses: snyk/actions/iac@master
+        env:
+          # In order to use the Snyk Action you will need to have a Snyk API token.
+          # More details in https://github.com/snyk/actions#getting-your-snyk-token
+          # or you can signup for free at https://snyk.io/login
+          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
+        with:
+          # Add the path to the configuration file that you would like to test.
+          # For example `deployment.yaml` for a Kubernetes deployment manifest
+          # or `main.tf` for a Terraform configuration file
+          file: your-file-to-test.yaml
+      - name: Upload result to GitHub Code Scanning
+        uses: github/codeql-action/upload-sarif@v1
+        with:
+          sarif_file: snyk.sarif

.gitignore

@@ -18,3 +18,14 @@ functions/generated_apps
 
 backend/onprem/app_sdk/apps
 *test.py
+
+*.exe
+*debug*
+
+shuffle-database/batch_metrics_enabled.conf
+shuffle-database/logging_enabled.conf
+shuffle-database/nodes
+shuffle-database/performance_analyzer_enabled.conf
+shuffle-database/rca_enabled.conf
+
+*/package-lock.json

README.md

@@ -1,20 +1,37 @@
-# Shuffle 
+<h1 align="center">
-[Shuffle](https://shuffler.io) is an automation platform to unify your security services (SOAR). It has thousands of premade integrations and is based on open frameworks like OpenAPI and Mitre Att&ck. The workflow editor is based on a no-code thought process to empower non-developers, and the app creator makes you able to integrate any platform in minutes.
 
-[![Discord](https://img.shields.io/discord/463752820026376202.svg?label=&logo=discord&logoColor=ffffff&color=7389D8&labelColor=6A7EC2)](https://discord.gg/B2CBzUm)
+[![Shuffle Logo](https://github.com/frikky/Shuffle/blob/launch/frontend/public/images/Shuffle_logo_new.png)](https://shuffler.io)
 
-![Example Shuffle webhook integration](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_webhook.png)
+Shuffle Automation
+
+</h1><h4 align="center">
+
+[Shuffle](https://shuffler.io) is an automation platform for and by the community, focusing on accessibility for anyone to automate. Security operations is complex, but it doesn't have to be.
+
+[_Key Features_](https://shuffler.io/docs/features) —
+[_Community & Support_](https://discord.gg/B2CBzUm) —
+[_Documentation_](https://shuffler.io/docs) —
+[_Getting Started_](https://shuffler.io/docs/getting_started) —
+[_Development_](https://github.com/frikky/Shuffle/blob/master/.github/CONTRIBUTING.md) 
+
+Follow us on Twitter at [@shuffleio](https://twitter.com/shuffleio).
+
+</h4>
+
+![Example Shuffle webhook integration](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/github_shuffle_img.png)
 
 ## Try it
-* Self-hosted: Check out the [installation guide](https://github.com/frikky/shuffle/blob/master/install-guide.md)
+* Self-hosted: Check out the [installation guide](https://github.com/frikky/shuffle/blob/master/.github/install-guide.md)
 * Cloud: Register at https://shuffler.io/register and get cooking (missing a lot of features)
 
 Please consider [sponsoring](https://github.com/sponsors/frikky) the project if you want to see more rapid development.
 
 ## Support
 * [Discord](https://discord.gg/B2CBzUm)
+* [Twitter](https://twitter.com/shuffleio)
 * [Email](mailto:frikky@shuffler.io)
 * [Open issue](https://github.com/frikky/Shuffle/issues/new)
+* [Shuffler.io](https://shuffler.io/contact)
 
 ## Blogposts
 * [1. Introducing Shuffle](https://medium.com/security-operation-capybara/introducing-shuffle-an-open-source-soar-platform-part-1-58a529de7d12)
@@ -23,25 +40,36 @@ Please consider [sponsoring](https://github.com/sponsors/frikky) the project if
 * [4. Real-time executions with TheHive, Cortex and MISP](https://medium.com/@Frikkylikeme/indicators-and-webhooks-with-thehive-cortex-and-misp-open-source-soar-part-4-f70cde942e59)
 
 ## Documentation
-[Documentation](https://shuffler.io/docs) can be found on https://shuffler.io/docs and is written in https://github.com/frikky/shuffle-docs.
+[Documentation](https://shuffler.io/docs) can be found on [https://shuffler.io/docs](https://shuffler.io/docs) and is written here: [https://github.com/frikky/shuffle-docs](https://github.com/frikky/shuffle-docs).
 
 ## Related repositories
-* Apps: https://github.com/frikky/shuffle-apps
+* OpenAPI apps: [https://github.com/frikky/security-openapis](https://github.com/frikky/security-openapis)
-* Workflows: https://github.com/frikky/shuffle-workflows
+* Documentation: [https://github.com/frikky/shuffle-docs](https://github.com/frikky/shuffle-docs)
-* Security OpenAPI apps: https://github.com/frikky/security-openapis
+* Workflows: [https://github.com/frikky/shuffle-workflows](https://github.com/frikky/shuffle-workflows)
-* Documentation: https://github.com/frikky/shuffle-docs
+* Python apps: [https://github.com/frikky/shuffle-apps](https://github.com/frikky/shuffle-apps)
 
 ## Features
-* Simple workflow automation editor 
+* Simple, feature rich [workflow editor](https://shuffler.io/docs/workflows)
-* Premade apps for a number of security tools
+* App creator using [OpenAPI](https://github.com/frikky/OpenAPI-security-definitions)
-* App creator for [OpenAPI](https://github.com/frikky/OpenAPI-security-definitions)
+* Premade apps for your security tools
-* Easy to learn Python library for custom apps
+* Organization and sub-organization control
-
+* Hybrid resource sharing with shuffler.io (optional)
-## Architecture
-![Shuffle Architecture](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_architecture.png)
 
 ## Website
-https://shuffler.io
+[https://shuffler.io](https://shuffler.io)
+
+## Contributing
+We want to make the world of cybersecurity more accessible and need all the help we can get. Send an email to [frikky@shuffler](mailto:frikky@shuffler.io) and we'll make sure to give you any training you may need.
+
+These are the main areas to contribute in:
+* Frontend (ReactJS)
+* Backend (Golang)
+* App Creation (Python & GUI w/OpenAPI)
+* Documentation (Markdown)
+* Workflow creation (GUI & Conceptualizing) 
+* Content Creation (Blogs, videos etc) 
+
+Contributing guidelines are outlined [here](https://github.com/frikky/Shuffle/blob/master/.github/CONTRIBUTING.md).
 
 ## Contributors 
 ![ICPL logo](https://github.com/frikky/Shuffle/blob/launch/frontend/src/assets/img/icpl_logo.png)
@@ -59,8 +87,13 @@ https://shuffler.io
 ## License
 All modular information related to Shuffle will be under MIT (anyone can use it for whatever purpose), with Shuffle itself using AGPLv3. 
 
-Apps & App SDK: MIT
+Workflows: MIT
+Documentation: MIT
 Shuffle backend: AGPLv3 
+Apps, specification and App SDK: MIT
+
+## Architecture
+![Shuffle Architecture](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_architecture.png)
 
 ### Repository overview 
 Below is the folder structure with a short explanation
@@ -68,17 +101,14 @@ Below is the folder structure with a short explanation
 ├── README.md				# What you're reading right now
 ├── backend					# Contains backend related code.
 │   ├── go-app 			# The backend golang webserver
-│   ├── app_gen 		# Code for app generation outside the Shuffle platform
 │   └── app_sdk			# The SDK used for apps
-├── frontend				# Contains frontend code. ReactJS and cytoscape. Horrible code :)
+├── frontend				# Contains frontend code. ReactJS, Material UI and cytoscape
-├── functions				# Contains google cloud function code mainly.
+├── functions				# Has execution and extension resources, such as the Wazuh integration
-│   ├── static_baseline.py	# Static code used by stitcher.go to generate code
-│   ├── stitcher.go		# Attempts to stitch together an app - part of backend now
 │   ├── onprem				# Code for onprem solutions
 │   │   ├── Orborus 	# Distributes execution locations
 │   │   ├── Worker		# Runs a workflow
 └ docker-compose.yml 	# Used for deployments
 ```
 
-**It's in BETA** - [Get in touch](https://shuffler.io/contact), send a mail to [frikky@shuffler.io](mailto:frikky@shuffler.io) or poke me on twitter [@frikkylikeme](https://twitter.com/frikkylikeme)
+**It's in BETA (0.8.60)** - [Get in touch](https://shuffler.io/contact), send a mail to [frikky@shuffler.io](mailto:frikky@shuffler.io) or poke me on twitter [@frikkylikeme](https://twitter.com/frikkylikeme)
 

SECURITY.md

@@ -0,0 +1,19 @@
+# Security Policy
+
+## Supported Versions
+
+Shuffle is currently still in beta, but we aim to support older version with critical severity issues, but do advise you to stay up to date with Major versions. 
+
+| Version | Supported          |
+| ------- | ------------------ |
+| >=0.9.0   | :white_check_mark: |
+| < 0.9.0   | :x:                |
+
+## Reporting a Vulnerability
+
+Reporting a vulnerability can either be done to (frikky@shuffler.io)[mailto:frikky@shuffler.io] or [through the contact page on our website](https://shuffler.io/contact)
+
+Security.txt: https://shuffler.io/.well_known/security.txt
+
+When a >medium severity vulnerability is discovered, expect it to be fixed ASAP - please nag us until it is. Security is a top priority, and we expect you to keep us accountable.
+In the case it makes sense, we'll further create a security advisory, and publish a new CVE for your new glorious finding.

backend/Dockerfile

@@ -1,4 +1,4 @@
-from golang as builder
+FROM golang:1.17.2-buster as builder
 
 # Add files
 RUN mkdir /app
@@ -7,18 +7,11 @@ WORKDIR /app
 ADD ./go-app/main.go /app
 ADD ./go-app/walkoff.go /app
 ADD ./go-app/docker.go /app
-ADD ./go-app/codegen.go /app
-ADD ./go-app/files.go /app
 
 ADD ./go-app/go.mod /app
 
 # Required files for code generation
 ADD ./app_sdk/app_base.py /app_sdk
-ADD ./app_sdk/static_baseline.py /app_sdk
-ADD ./app_sdk_kali/app_base.py /app_sdk_kali
-ADD ./app_sdk_kali/static_baseline.py /app_sdk_kali
-ADD ./app_sdk_blackarch/app_base.py /app_sdk_blackarch
-ADD ./app_sdk_blackarch/static_baseline.py /app_sdk_blackarch
 ADD ./app_gen /app_gen
 
 RUN go get -v
@@ -29,12 +22,10 @@ RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o webapp .
 FROM alpine:latest as certs
 RUN apk --update add ca-certificates
 
-FROM alpine:3.12
+FROM alpine:3.14.2
 
 COPY --from=builder /app/ /app
 COPY --from=builder /app_sdk/ /app_sdk
-COPY --from=builder /app_sdk_kali/ /app_sdk_kali
-COPY --from=builder /app_sdk_blackarch/ /app_sdk_blackarch
 COPY --from=builder /app_gen/ /app_gen
 COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
 

backend/app_gen/openapi/baseline/Dockerfile

@@ -2,24 +2,27 @@
 FROM frikky/shuffle:app_sdk as base
 
 # We're going to stage away all of the bloat from the build tools so lets create a builder stage
-FROM base as builder
+#FROM base as builder
 
 # Install all alpine build tools needed for our pip installs
-RUN apk --no-cache add --update alpine-sdk libffi libffi-dev musl-dev openssl-dev 
+#RUN apk --no-cache add --update alpine-sdk libffi libffi-dev musl-dev openssl-dev 
 
 # Install all of our pip packages in a single directory that we can copy to our base image later
-RUN mkdir /install
+#RUN mkdir /install
-WORKDIR /install
+#WORKDIR /install
-COPY requirements.txt /requirements.txt
+#COPY requirements.txt /requirements.txt
-RUN pip install --prefix="/install" -r /requirements.txt
+#
+## Switch back to our base image and copy in all of our built packages and source code
+#FROM base
+#COPY --from=builder /install /usr/local
 
-# Switch back to our base image and copy in all of our built packages and source code
-FROM base
-COPY --from=builder /install /usr/local
-COPY src /app
 
 # Install any binary dependencies needed in our final image - this can be a lot of different stuff
-RUN apk --no-cache add --update libmagic
+#RUN apk --no-cache add --update libmagic
+WORKDIR /
+COPY requirements.txt /requirements.txt
+RUN pip install --prefix="/usr/local" -r /requirements.txt
+COPY src /app
 
 # Finally, lets run our app!
 WORKDIR /app

backend/app_sdk/Dockerfile

@@ -1,16 +1,20 @@
-FROM python:3.7-alpine as base
+#FROM python:3.9.1-alpine as base
+FROM python:3.10.0-alpine as base
 
 FROM base as builder
-RUN apk --no-cache add --update alpine-sdk libffi libffi-dev musl-dev openssl-dev
+RUN apk --no-cache add --update alpine-sdk libffi libffi-dev musl-dev openssl-dev tzdata coreutils
 
 RUN mkdir /install
 WORKDIR /install
 
+FROM base
+
+#--no-cache 
+RUN apk update && apk add --update tzdata libmagic alpine-sdk libffi libffi-dev musl-dev openssl-dev coreutils
+
+COPY --from=builder /install /usr/local
 COPY requirements.txt /requirements.txt
 RUN pip3 install -r /requirements.txt
 
-FROM base
-
-COPY --from=builder /install /usr/local
 COPY __init__.py /app/walkoff_app_sdk/__init__.py
 COPY app_base.py /app/walkoff_app_sdk/app_base.py

backend/app_sdk/Dockerfile_alpine_grpc

@@ -0,0 +1,42 @@
+FROM python:3.10.0-alpine as base
+
+FROM base as builder
+RUN apk --no-cache add --update \
+        alpine-sdk \
+        build-base  \
+        g++ \
+        gcc \
+        libffi \
+        libffi-dev \
+        libstdc++ \
+        linux-headers \
+        musl-dev \
+        openssl-dev \
+        tzdata \
+        coreutils
+
+RUN pip install --upgrade pip && \
+    pip install --prefix="/install" --no-cache-dir grpcio grpcio-tools && \
+    apk del --purge \
+    g++ \
+    gcc \
+    musl-dev \
+    libffi-dev \
+    libstdc++ \
+    build-base \
+    linux-headers
+
+RUN mkdir -p /install
+WORKDIR /install
+
+FROM base
+
+#--no-cache
+RUN apk update && apk add --update tzdata libmagic alpine-sdk libffi libffi-dev musl-dev openssl-dev coreutils
+
+COPY --from=builder /install /usr/local
+COPY requirements.txt /requirements.txt
+RUN pip3 install -r /requirements.txt
+
+COPY __init__.py /app/walkoff_app_sdk/__init__.py
+COPY app_base.py /app/walkoff_app_sdk/app_base.py

backend/app_sdk/Dockerfile_ubuntu

@@ -0,0 +1,22 @@
+FROM ubuntu as base
+
+FROM base as builder
+
+RUN apt-get update
+RUN apt-get dist-upgrade -y
+RUN apt install build-essential libffi-dev musl-dev openssl python3 python3-pip -y
+
+RUN mkdir /install
+WORKDIR /install
+
+COPY requirements.txt /requirements.txt
+RUN pip install --prefix="/install" -r /requirements.txt
+
+FROM base
+RUN apt-get update
+RUN apt-get dist-upgrade -y
+RUN apt install build-essential libffi-dev musl-dev openssl python3 python3-pip -y
+
+COPY --from=builder /install /usr/local
+COPY __init__.py /app/walkoff_app_sdk/__init__.py
+COPY app_base.py /app/walkoff_app_sdk/app_base.py

backend/app_sdk/README.md

@@ -1,9 +1,5 @@
 # app_sdk.py
 This is the SDK used for apps to behave like they should. 
-To change it in the backend, upload it to Buckets/shuffler.appspot.com/generated_apps/baseline.
-
-# static_baseline.py
-It's used for python code generation and should be under MIT. Has to be located here because it's used by the backend.
 
 ## If you want to update apps.. PS: downloads from docker hub do overrides.. :)
 1. Write your code & check if runtime works
@@ -12,5 +8,15 @@ It's used for python code generation and should be under MIT. Has to be located
 4. Delete the specific app's Docker image (docker rmi frikky/shuffle:...)
 5. Rebuild the Docker image (click load in GUI?)
 
+## Cloud updates
+1. Go to shuffle cloud on GCP
+2. Go to Cloud Storage
+3. Find shuffler.appspot.com
+4. Navigate to generated_apps/baseline
+5. Update SDK there. This will make all new apps run with the new SDK
+
+## Cloud app force-updates
+1. Run the "stitcher.go" program in the public shuffle-shared repository.
+
 # LICENSE 
 Everything in here is MIT, not AGPLv3 as indicated by the license.

backend/app_sdk/build.sh

@@ -1,14 +1,46 @@
 #!/bin/bash
+
+### DEFAULT 
 NAME=shuffle-app_sdk
-VERSION=0.8.54
+VERSION=0.9.70
 
 docker rmi docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION --force
-docker build . -t frikky/shuffle:app_sdk -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
+docker build . -f Dockerfile -t frikky/shuffle:app_sdk -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION -t ghcr.io/frikky/$NAME:nightly
-
-#docker push frikky/$NAME:$VERSION
-#docker push docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION
-#docker push ghcr.io/frikky/$NAME:$VERSION
-#docker tag ghcr.io/frikky/$NAME:$VERSION frikky/shuffle:app_sdk
 
 docker push frikky/shuffle:app_sdk
 docker push ghcr.io/frikky/$NAME:$VERSION
+docker push ghcr.io/frikky/$NAME:nightly
+docker push ghcr.io/frikky/$NAME:latest
+
+
+
+
+#### UBUNTU
+NAME=shuffle-app_sdk_ubuntu
+docker build . -f Dockerfile_ubuntu -t frikky/shuffle:app_sdk_ubuntu -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
+docker push frikky/shuffle:app_sdk_ubuntu
+docker push ghcr.io/frikky/$NAME:$VERSION
+
+#### Alpine GRPC 
+NAME=shuffle-app_sdk_grpc
+docker build . -f Dockerfile_alpine_grpc -t frikky/shuffle:app_sdk_grpc -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
+docker push frikky/shuffle:app_sdk_grpc
+docker push ghcr.io/frikky/$NAME:$VERSION
+
+
+
+#### KALI ###
+#NAME=shuffle-app_sdk_kali
+#docker build . -f Dockerfile_kali -t frikky/shuffle:app_sdk_kali -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
+#
+#docker push frikky/shuffle:app_sdk_kali
+#docker push ghcr.io/frikky/$NAME:$VERSION
+#docker push ghcr.io/frikky/$NAME:nightly
+
+### BLACKARCH ###
+#NAME=shuffle-app_sdk_blackarch
+#docker build . -f Dockerfile_blackarch -t frikky/shuffle:app_sdk_blackarch -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
+#
+#docker push frikky/shuffle:app_sdk_blackarch
+#docker push ghcr.io/frikky/$NAME:$VERSION
+#docker push ghcr.io/frikky/$NAME:nightly

backend/app_sdk/requirements.txt

@@ -1,2 +1,7 @@
-urllib3
+urllib3==1.26.5
-requests
+requests==2.25.1
+MarkupSafe==2.0.1
+liquidpy==0.7.3
+flask[async]==2.0.2
+waitress==2.1.0
+#flask==1.1.2

backend/app_sdk/static_baseline.py

@@ -1,76 +0,0 @@
-import os
-import sys
-import time 
-import logging
-import requests
-
-# Goal here:
-# * Make an app from WALKOFF able to run without app_base.py from WALKOFF
-# # How:
-# * Make it rely 100% on INPUT throug HTTP invocations instead of redis READS 
-# # But really, how?
-# * Make a WORKER that reads the queue, and reuses a function 
-
-# Here to get it global
-apikey = ""
-try:
-    apikey = os.environ["FUNCTION_APIKEY"]
-except KeyError:
-    pass
-
-# Authorize the execution 
-def authorization(request):
-    # This is basically my issue, but it enforces the use of an internal API key for execution
-    try:
-        apikey = os.environ["FUNCTION_APIKEY"]
-    except KeyError:
-        return f"Internal server error", 500
-
-
-    # Check API key from ENV authentication
-    authentication = request.headers.get("Authorization")
-    if authentication == None: return f"Unauthorized", 401
-
-    apikey_split = authentication.split(" ")
-    if apikey_split[0] != "Bearer" or len(apikey_split) != 2:
-        return f"Apikey error", 401
-
-    if apikey != apikey_split[1]:
-        return f"Unauthorized", 401
-
-    return run(request)
-
-class AppBase:
-    """ The base class for Python-based Walkoff applications, handles Redis and logging configurations. """
-    __version__ = None
-    app_name = None
-
-    def __init__(self, redis=None, logger=None, console_logger=None):#, docker_client=None):
-        self.logger = logger if logger is not None else logging.getLogger("AppBaseLogger")
-        self.redis=redis
-        self.console_logger=console_logger 
-        self.current_execution_id = None
-        self.url = "https://shuffler.io"
-        self.apikey = apikey
-
-    @classmethod
-    async def run(cls, action):
-        """ Connect to Redis and HTTP session, await actions """
-        logging.basicConfig(format="{asctime} - {name} - {levelname}:{message}", style='{')
-        logger = logging.getLogger(f"{cls.__name__}")
-        logger.setLevel(logging.DEBUG)
-
-        app = cls(redis=None, logger=logger, console_logger=logger)
-
-        # Authorization for the app/function to control the workflow
-        # Function will crash if its wrong, which it probably should. 
-
-        await app.execute_action(action)
-    
-    async def execute_action(self, action):
-        # FIXME - add request for the function STARTING here. Use "results stream" or something
-        # PAUSED, AWAITING_DATA, PENDING, COMPLETED, ABORTED, EXECUTING, SUCCESS, FAILURE
-
-        self.authorization = action["authorization"]
-        self.execution_id = action["execution_id"]
-        self.current_execution_id = action["execution_id"]

backend/app_sdk_blackarch/LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2020 Frikkylikeme
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

backend/app_sdk_blackarch/README.md

@@ -1,16 +0,0 @@
-# app_sdk.py
-This is the SDK used for apps to behave like they should. 
-To change it in the backend, upload it to Buckets/shuffler.appspot.com/generated_apps/baseline.
-
-# static_baseline.py
-It's used for python code generation and should be under MIT. Has to be located here because it's used by the backend.
-
-## If you want to update apps.. PS: downloads from docker hub do overrides.. :)
-1. Write your code & check if runtime works
-2. Build app_base image
-3. docker rm $(docker ps -aq) # Remove all stopped containers
-4. Delete the specific app's Docker image (docker rmi frikky/shuffle:...)
-5. Rebuild the Docker image (click load in GUI?)
-
-# LICENSE 
-Everything in here is MIT, not AGPLv3 as indicated by the license.

backend/app_sdk_blackarch/build.sh

@@ -1,13 +0,0 @@
-#!/bin/bash
-NAME=app_sdk_blackarch
-VERSION=0.7.3
-
-docker rmi docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION --force
-docker build . -t frikky/shuffle:$NAME -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
-
-#docker push frikky/$NAME:$VERSION
-#docker push docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION
-#docker push ghcr.io/frikky/$NAME:$VERSION
-
-docker push frikky/shuffle:$NAME
-docker push ghcr.io/frikky/$NAME:$VERSION

backend/app_sdk_blackarch/static_baseline.py

@@ -1,76 +0,0 @@
-import os
-import sys
-import time 
-import logging
-import requests
-
-# Goal here:
-# * Make an app from WALKOFF able to run without app_base.py from WALKOFF
-# # How:
-# * Make it rely 100% on INPUT throug HTTP invocations instead of redis READS 
-# # But really, how?
-# * Make a WORKER that reads the queue, and reuses a function 
-
-# Here to get it global
-apikey = ""
-try:
-    apikey = os.environ["FUNCTION_APIKEY"]
-except KeyError:
-    pass
-
-# Authorize the execution 
-def authorization(request):
-    # This is basically my issue, but it enforces the use of an internal API key for execution
-    try:
-        apikey = os.environ["FUNCTION_APIKEY"]
-    except KeyError:
-        return f"Internal server error", 500
-
-
-    # Check API key from ENV authentication
-    authentication = request.headers.get("Authorization")
-    if authentication == None: return f"Unauthorized", 401
-
-    apikey_split = authentication.split(" ")
-    if apikey_split[0] != "Bearer" or len(apikey_split) != 2:
-        return f"Apikey error", 401
-
-    if apikey != apikey_split[1]:
-        return f"Unauthorized", 401
-
-    return run(request)
-
-class AppBase:
-    """ The base class for Python-based Walkoff applications, handles Redis and logging configurations. """
-    __version__ = None
-    app_name = None
-
-    def __init__(self, redis=None, logger=None, console_logger=None):#, docker_client=None):
-        self.logger = logger if logger is not None else logging.getLogger("AppBaseLogger")
-        self.redis=redis
-        self.console_logger=console_logger 
-        self.current_execution_id = None
-        self.url = "https://shuffler.io"
-        self.apikey = apikey
-
-    @classmethod
-    async def run(cls, action):
-        """ Connect to Redis and HTTP session, await actions """
-        logging.basicConfig(format="{asctime} - {name} - {levelname}:{message}", style='{')
-        logger = logging.getLogger(f"{cls.__name__}")
-        logger.setLevel(logging.DEBUG)
-
-        app = cls(redis=None, logger=logger, console_logger=logger)
-
-        # Authorization for the app/function to control the workflow
-        # Function will crash if its wrong, which it probably should. 
-
-        await app.execute_action(action)
-    
-    async def execute_action(self, action):
-        # FIXME - add request for the function STARTING here. Use "results stream" or something
-        # PAUSED, AWAITING_DATA, PENDING, COMPLETED, ABORTED, EXECUTING, SUCCESS, FAILURE
-
-        self.authorization = action["authorization"]
-        self.execution_id = action["execution_id"]
-        self.current_execution_id = action["execution_id"]

backend/app_sdk_kali/LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2020 Frikkylikeme
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

backend/app_sdk_kali/README.md

@@ -1,16 +0,0 @@
-# app_sdk.py
-This is the SDK used for apps to behave like they should. 
-To change it in the backend, upload it to Buckets/shuffler.appspot.com/generated_apps/baseline.
-
-# static_baseline.py
-It's used for python code generation and should be under MIT. Has to be located here because it's used by the backend.
-
-## If you want to update apps.. PS: downloads from docker hub do overrides.. :)
-1. Write your code & check if runtime works
-2. Build app_base image
-3. docker rm $(docker ps -aq) # Remove all stopped containers
-4. Delete the specific app's Docker image (docker rmi frikky/shuffle:...)
-5. Rebuild the Docker image (click load in GUI?)
-
-# LICENSE 
-Everything in here is MIT, not AGPLv3 as indicated by the license.

backend/app_sdk_kali/build.sh

@@ -1,13 +0,0 @@
-#!/bin/bash
-NAME=app_sdk_kali
-VERSION=0.7.3
-
-docker rmi docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION --force
-docker build . -t frikky/shuffle:$NAME -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
-
-#docker push frikky/$NAME:$VERSION
-#docker push docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION
-#docker push ghcr.io/frikky/$NAME:$VERSION
-
-docker push frikky/shuffle:$NAME
-docker push ghcr.io/frikky/$NAME:$VERSION

backend/app_sdk_kali/requirements.txt

@@ -1,2 +0,0 @@
-requests
-urllib3

backend/app_sdk_kali/static_baseline.py

@@ -1,76 +0,0 @@
-import os
-import sys
-import time 
-import logging
-import requests
-
-# Goal here:
-# * Make an app from WALKOFF able to run without app_base.py from WALKOFF
-# # How:
-# * Make it rely 100% on INPUT throug HTTP invocations instead of redis READS 
-# # But really, how?
-# * Make a WORKER that reads the queue, and reuses a function 
-
-# Here to get it global
-apikey = ""
-try:
-    apikey = os.environ["FUNCTION_APIKEY"]
-except KeyError:
-    pass
-
-# Authorize the execution 
-def authorization(request):
-    # This is basically my issue, but it enforces the use of an internal API key for execution
-    try:
-        apikey = os.environ["FUNCTION_APIKEY"]
-    except KeyError:
-        return f"Internal server error", 500
-
-
-    # Check API key from ENV authentication
-    authentication = request.headers.get("Authorization")
-    if authentication == None: return f"Unauthorized", 401
-
-    apikey_split = authentication.split(" ")
-    if apikey_split[0] != "Bearer" or len(apikey_split) != 2:
-        return f"Apikey error", 401
-
-    if apikey != apikey_split[1]:
-        return f"Unauthorized", 401
-
-    return run(request)
-
-class AppBase:
-    """ The base class for Python-based Walkoff applications, handles Redis and logging configurations. """
-    __version__ = None
-    app_name = None
-
-    def __init__(self, redis=None, logger=None, console_logger=None):#, docker_client=None):
-        self.logger = logger if logger is not None else logging.getLogger("AppBaseLogger")
-        self.redis=redis
-        self.console_logger=console_logger 
-        self.current_execution_id = None
-        self.url = "https://shuffler.io"
-        self.apikey = apikey
-
-    @classmethod
-    async def run(cls, action):
-        """ Connect to Redis and HTTP session, await actions """
-        logging.basicConfig(format="{asctime} - {name} - {levelname}:{message}", style='{')
-        logger = logging.getLogger(f"{cls.__name__}")
-        logger.setLevel(logging.DEBUG)
-
-        app = cls(redis=None, logger=logger, console_logger=logger)
-
-        # Authorization for the app/function to control the workflow
-        # Function will crash if its wrong, which it probably should. 
-
-        await app.execute_action(action)
-    
-    async def execute_action(self, action):
-        # FIXME - add request for the function STARTING here. Use "results stream" or something
-        # PAUSED, AWAITING_DATA, PENDING, COMPLETED, ABORTED, EXECUTING, SUCCESS, FAILURE
-
-        self.authorization = action["authorization"]
-        self.execution_id = action["execution_id"]
-        self.current_execution_id = action["execution_id"]

backend/database/Dockerfile

@@ -1,5 +0,0 @@
-# 2. docker run -p 8000:8000 
-FROM google/cloud-sdk 
-
-EXPOSE 8000
-CMD ["gcloud", "beta", "emulators", "datastore", "start", "--project=shuffle", "--host-port", "0.0.0.0:8000", "--data-dir=/etc/shuffle"]

backend/database/opensearch/docker-compose.yml

@@ -0,0 +1,34 @@
+version: '3'
+services:
+  opensearch-node1:
+    image: opensearchproject/opensearch:latest
+    hostname: shuffle-database
+    container_name: shuffle-opensearch
+    environment:
+      - cluster.name=shuffle-cluster
+      - node.name=shuffle-opensearch
+      - discovery.seed_hosts=shuffle-opensearch
+      - cluster.initial_master_nodes=shuffle-opensearch
+      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
+      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
+      - cluster.routing.allocation.disk.threshold_enabled=false
+      - opendistro_security.disabled=true
+    ulimits:
+      memlock:
+        soft: -1
+        hard: -1
+      nofile:
+        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
+        hard: 65536
+    volumes:
+      - ~/git/shuffle/shuffle-database:/usr/share/opensearch/data
+    ports:
+      - 9200:9200
+    networks:
+      - opensearch-net
+
+volumes:
+  opensearch-data1:
+
+networks:
+  opensearch-net:

backend/go-app/app.yaml

@@ -1,98 +0,0 @@
-runtime: go111
-
-env_variables:
-
-automatic_scaling:
-  max_instances: 1
-  min_instances: 1
-
-handlers:
-- url: /api/(.*)
-  script: auto
-  secure: always
-- url: /static/js/(.*)
-  static_files: build/static/js/\1
-  upload: build/static/js/(.*)
-  secure: always
-- url: /static/css/(.*)
-  static_files: build/static/css/\1
-  upload: build/static/css/(.*)
-  secure: always
-- url: /images/(.*)
-  static_files: build/images/\1
-  upload: build/images/(.*)
-  secure: always
-- url: /(.*\.(json|ico))$
-  static_files: build/\1
-  upload: build/.*\.(json|ico)$
-  secure: always
-- url: /manifest.json
-  static_files: build/manifest.json
-  upload: build/manifest.json
-  secure: always
-
-# lol.. wildcard doesn't work with /api/(.*) for some reason
-- url: /
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /home
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /passwordreset
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /login
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /register
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /workflows
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /workflows/(.*)
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /info/(.*)
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /docs/(.*)
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /docs
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /settings
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /apps
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /contact
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /apps/(.*)
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /register/(.*)
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always
-- url: /passwordreset/(.*)
-  static_files: build/index.html
-  upload: build/index.html
-  secure: always

backend/go-app/docker.go

@@ -2,21 +2,28 @@ package main
 
 // Docker
 import (
+	"github.com/shuffle/shuffle-shared"
+
 	"archive/tar"
+	//"bufio"
 	"path/filepath"
+	//"strconv"
 
 	"bytes"
 	"context"
 	"encoding/json"
 	"errors"
 	"fmt"
+
+	//"github.com/docker/docker"
 	"github.com/docker/docker/api/types"
-	"github.com/docker/docker/api/types/container"
+	//"github.com/docker/docker/api/types/container"
 	"github.com/docker/docker/client"
+	newdockerclient "github.com/fsouza/go-dockerclient"
 	"github.com/go-git/go-billy/v5"
 
-	network "github.com/docker/docker/api/types/network"
+	//network "github.com/docker/docker/api/types/network"
-	natting "github.com/docker/go-connections/nat"
+	//natting "github.com/docker/go-connections/nat"
 
 	"io"
 	"io/ioutil"
@@ -24,7 +31,6 @@ import (
 	"net/http"
 	"os"
 	"strings"
-	//"google.golang.org/appengine"
 )
 
 // Parses a directory with a Dockerfile into a tar for Docker images..
@@ -213,11 +219,27 @@ func buildImageMemory(fs billy.Filesystem, tags []string, dockerfileFolder strin
 
 	// Dockerfile is inside the TAR itself. Not local context
 	// docker build --build-arg http_proxy=http://my.proxy.url
+	// Attempt at setting name according to #359: https://github.com/frikky/Shuffle/issues/359
+	labels := map[string]string{}
+	//target := ""
+	//if len(tags) > 0 {
+	//	if strings.Contains(tags[0], ":") {
+	//		version := strings.Split(tags[0], ":")
+	//		if len(version) == 2 {
+	//			target = fmt.Sprintf("shuffle-build-%s", version[1])
+	//			tags = append(tags, target)
+	//			labels["name"] = target
+	//		}
+	//	}
+	//}
+
 	buildOptions := types.ImageBuildOptions{
 		Remove:    true,
 		Tags:      tags,
 		BuildArgs: map[string]*string{},
+		Labels:    labels,
 	}
+
 	// NetworkMode: "host",
 
 	httpProxy := os.Getenv("HTTP_PROXY")
@@ -230,55 +252,59 @@ func buildImageMemory(fs billy.Filesystem, tags []string, dockerfileFolder strin
 	}
 
 	// Build the actual image
-	log.Printf("[INFO] Building %s. This may take up to a few minutes.", dockerfileFolder)
+	log.Printf(`[INFO] Building %s with proxy "%s". Tags: "%s". This may take up to a few minutes.`, dockerfileFolder, httpsProxy, strings.Join(tags, ","))
 	imageBuildResponse, err := client.ImageBuild(
 		ctx,
 		dockerFileTarReader,
 		buildOptions,
 	)
 
+	//log.Printf("RESPONSE: %#v", imageBuildResponse)
 	//log.Printf("Response: %#v", imageBuildResponse.Body)
-	//log.Printf("IMAGERESPONSE: %#v", imageBuildResponse.Body)
+	//log.Printf("[DEBUG] IMAGERESPONSE: %#v", imageBuildResponse.Body)
 
-	defer imageBuildResponse.Body.Close()
+	if imageBuildResponse.Body != nil {
-	buildBuf := new(strings.Builder)
+		defer imageBuildResponse.Body.Close()
-	_, newerr := io.Copy(buildBuf, imageBuildResponse.Body)
+		buildBuf := new(strings.Builder)
-	if newerr != nil {
+		_, newerr := io.Copy(buildBuf, imageBuildResponse.Body)
-		log.Printf("Failed reading Docker build STDOUT: %s", newerr)
+		if newerr != nil {
-	} else {
+			log.Printf("[WARNING] Failed reading Docker build STDOUT: %s", newerr)
-		log.Printf("STRING: %s", buildBuf.String())
+		} else {
-		if strings.Contains(buildBuf.String(), "errorDetail") {
+			log.Printf("[INFO] STRING: %s", buildBuf.String())
-			log.Printf("[ERROR] Docker build:\n%s\nERROR ABOVE: Trying to pull tags from: %s", buildBuf.String(), strings.Join(tags, "\n"))
+			if strings.Contains(buildBuf.String(), "errorDetail") {
+				log.Printf("[ERROR] Docker build:\n%s\nERROR ABOVE: Trying to pull tags from: %s", buildBuf.String(), strings.Join(tags, "\n"))
 
-			// Handles pulling of the same image if applicable
+				// Handles pulling of the same image if applicable
-			// This fixes some issues with older versions of Docker which can't build
+				// This fixes some issues with older versions of Docker which can't build
-			// on their own ( <17.05 )
+				// on their own ( <17.05 )
-			pullOptions := types.ImagePullOptions{}
+				pullOptions := types.ImagePullOptions{}
-			downloaded := false
+				downloaded := false
-			for _, image := range tags {
+				for _, image := range tags {
-				// Is this ok? Not sure. Tags shouldn't be controlled here prolly.
+					// Is this ok? Not sure. Tags shouldn't be controlled here prolly.
-				image = strings.ToLower(image)
+					image = strings.ToLower(image)
 
-				newImage := fmt.Sprintf("%s/%s", registryName, image)
+					newImage := fmt.Sprintf("%s/%s", registryName, image)
-				log.Printf("[INFO] Pulling image %s", newImage)
+					log.Printf("[INFO] Pulling image %s", newImage)
-				reader, err := client.ImagePull(ctx, newImage, pullOptions)
+					reader, err := client.ImagePull(ctx, newImage, pullOptions)
-				if err != nil {
+					if err != nil {
-					log.Printf("[ERROR] Failed getting image %s: %s", newImage, err)
+						log.Printf("[ERROR] Failed getting image %s: %s", newImage, err)
-					continue
+						continue
+					}
+
+					// Attempt to retag the image to not contain registry...
+
+					//newBuf := buildBuf
+					downloaded = true
+					io.Copy(os.Stdout, reader)
+					log.Printf("[INFO] Successfully downloaded and built %s", newImage)
 				}
 
-				// Attempt to retag the image to not contain registry...
+				if !downloaded {
 
-				//newBuf := buildBuf
+					return errors.New(fmt.Sprintf("Failed to build / download images %s", strings.Join(tags, ",")))
-				downloaded = true
+				}
-				io.Copy(os.Stdout, reader)
+				//baseDockerName
-				log.Printf("[INFO] Successfully downloaded and built %s", newImage)
 			}
-
-			if !downloaded {
-				return errors.New(fmt.Sprintf("Failed to build / download images %s", strings.Join(tags, ",")))
-			}
-			//baseDockerName
 		}
 	}
 
@@ -299,7 +325,7 @@ func buildImage(tags []string, dockerfileFolder string) error {
 		return err
 	}
 
-	log.Printf("Tags: %s", tags)
+	log.Printf("[INFO] Docker Tags: %s", tags)
 	dockerfileSplit := strings.Split(dockerfileFolder, "/")
 
 	// Create a buffer
@@ -390,92 +416,9 @@ func stopWebhook(image string, identifier string) error {
 	return nil
 }
 
-// FIXME - remember to set DOCKER_API_VERSION
-// FIXME - remove github.com/docker/docker/vendor
-// FIXME - Library dependencies for NAT is fucked..
-// https://docs.docker.com/develop/sdk/examples/
-func deployWebhook(image string, identifier string, path string, port string, callbackurl string, apikey string) error {
-	cli, err := client.NewEnvClient()
-	if err != nil {
-		fmt.Println("Unable to create docker client")
-		return err
-	}
-
-	newport, err := natting.NewPort("tcp", port)
-	if err != nil {
-		fmt.Println("Unable to create docker port")
-		return err
-	}
-
-	// FIXME - logging?
-
-	hostConfig := &container.HostConfig{
-		PortBindings: natting.PortMap{
-			newport: []natting.PortBinding{
-				{
-					HostIP:   "0.0.0.0",
-					HostPort: port,
-				},
-			},
-		},
-		RestartPolicy: container.RestartPolicy{
-			Name: "always",
-		},
-		LogConfig: container.LogConfig{
-			Type:   "json-file",
-			Config: map[string]string{},
-		},
-	}
-
-	//networkConfig := &network.NetworkSettings{}
-	networkConfig := &network.NetworkingConfig{
-		EndpointsConfig: map[string]*network.EndpointSettings{},
-	}
-
-	test := &network.EndpointSettings{
-		Gateway: "helo",
-	}
-
-	networkConfig.EndpointsConfig["bridge"] = test
-
-	exposedPorts := map[natting.Port]struct{}{
-		newport: struct{}{},
-	}
-
-	config := &container.Config{
-		Image: image,
-		Env: []string{
-			fmt.Sprintf("URIPATH=%s", path),
-			fmt.Sprintf("HOOKPORT=%s", port),
-			fmt.Sprintf("CALLBACKURL=%s", callbackurl),
-			fmt.Sprintf("APIKEY=%s", apikey),
-			fmt.Sprintf("HOOKID=%s", identifier),
-		},
-		ExposedPorts: exposedPorts,
-		Hostname:     fmt.Sprintf("%s-%s", image, identifier),
-	}
-
-	cont, err := cli.ContainerCreate(
-		context.Background(),
-		config,
-		hostConfig,
-		networkConfig,
-		fmt.Sprintf("%s-%s", image, identifier),
-	)
-
-	if err != nil {
-		log.Println(err)
-		return err
-	}
-
-	cli.ContainerStart(context.Background(), cont.ID, types.ContainerStartOptions{})
-	log.Printf("Container %s is created", cont.ID)
-	return nil
-}
-
 // Starts a new webhook
 func handleStopHookDocker(resp http.ResponseWriter, request *http.Request) {
-	cors := handleCors(resp, request)
+	cors := shuffle.HandleCors(resp, request)
 	if cors {
 		return
 	}
@@ -500,7 +443,7 @@ func handleStopHookDocker(resp http.ResponseWriter, request *http.Request) {
 	}
 
 	ctx := context.Background()
-	hook, err := getHook(ctx, fileId)
+	hook, err := shuffle.GetHook(ctx, fileId)
 	if err != nil {
 		log.Printf("Failed getting hook %s (stop docker): %s", fileId, err)
 		resp.WriteHeader(401)
@@ -520,8 +463,8 @@ func handleStopHookDocker(resp http.ResponseWriter, request *http.Request) {
 
 	hook.Status = "stopped"
 	hook.Running = false
-	hook.Actions = []HookAction{}
+	hook.Actions = []shuffle.HookAction{}
-	err = setHook(ctx, *hook)
+	err = shuffle.SetHook(ctx, *hook)
 	if err != nil {
 		log.Printf("Failed setting hook: %s", err)
 		resp.WriteHeader(401)
@@ -560,7 +503,7 @@ var webhook = `{
 // Starts a new webhook
 func handleDeleteHookDocker(resp http.ResponseWriter, request *http.Request) {
 	ctx := context.Background()
-	cors := handleCors(resp, request)
+	cors := shuffle.HandleCors(resp, request)
 	if cors {
 		return
 	}
@@ -584,7 +527,7 @@ func handleDeleteHookDocker(resp http.ResponseWriter, request *http.Request) {
 		return
 	}
 
-	err := DeleteKey(ctx, "hooks", fileId)
+	err := shuffle.DeleteKey(ctx, "hooks", fileId)
 	if err != nil {
 		resp.WriteHeader(401)
 		resp.Write([]byte(`{"success": false, "message": "Can't delete"}`))
@@ -605,121 +548,6 @@ func handleDeleteHookDocker(resp http.ResponseWriter, request *http.Request) {
 	resp.Write([]byte(`{"success": true, "message": "Deleted webhook"}`))
 }
 
-// Starts a new webhook
-func handleStartHookDocker(resp http.ResponseWriter, request *http.Request) {
-	cors := handleCors(resp, request)
-	if cors {
-		return
-	}
-
-	location := strings.Split(request.URL.String(), "/")
-
-	var fileId string
-	if location[1] == "api" {
-		if len(location) <= 4 {
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-
-		fileId = location[4]
-	}
-
-	if len(fileId) != 32 {
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "message": "ID not valid"}`))
-		return
-	}
-
-	ctx := context.Background()
-	hook, err := getHook(ctx, fileId)
-	if err != nil {
-		log.Printf("Failed getting hook %s (start docker): %s", fileId, err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	if len(hook.Info.Url) == 0 {
-		log.Printf("Hook url can't be empty.")
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	log.Printf("Status: %s", hook.Status)
-	log.Printf("Running: %t", hook.Running)
-	if hook.Running || hook.Status == "Running" {
-		message := fmt.Sprintf("Error: %s is already running", hook.Id)
-		log.Println(message)
-		resp.WriteHeader(401)
-		resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "%s"}`, message)))
-		return
-	}
-
-	// FIXME - verify?
-	// FIXME - static port? Generate from available range.
-	image := "webhook"
-	filepath := "/webhook"
-	baseUrl := "http://localhost"
-	callbackUrl := "http://localhost:8001"
-
-	// This is here to force stop and remove the old webhook
-	err = stopWebhook(image, fileId)
-	if err != nil {
-		log.Printf("Container stop issue for %s-%s: %s", image, fileId, err)
-	}
-
-	// Dynamic ish ports
-	var startPort int64 = 5001
-	var endPort int64 = 5010
-	port := findAvailablePorts(startPort, endPort)
-	if len(port) == 0 {
-		message := fmt.Sprintf("Not ports available in the range %d-%d", startPort, endPort)
-		log.Println(message)
-		resp.WriteHeader(401)
-		resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "%s"}`, message)))
-		return
-
-	}
-
-	hook.Status = "running"
-	hook.Running = true
-
-	// Set this for more than just hooks?
-	if hook.Type == "webhook" {
-		hook.Info.Url = fmt.Sprintf("%s:%s%s", baseUrl, port, filepath)
-	}
-	err = setHook(ctx, *hook)
-	if err != nil {
-		log.Printf("Failed setting hook: %s", err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	// Cloud run? Let's make a generic webhook that can be deployed easily
-	log.Printf("Should run a webhook with the following: \nUrl: %s\nId: %s\n", hook.Info.Url, hook.Id)
-
-	// FIXME - set port based on what the user specified / what was generated
-	// FIXME - add nonstatic APIKEY
-	apiKey := "ASD"
-
-	err = deployWebhook(image, fileId, filepath, port, callbackUrl, apiKey)
-	if err != nil {
-		log.Printf("Failed starting container %s-%s: %s", image, fileId, err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	// FIXME - get some real data?
-	log.Printf("[INFO] Successfully started %s-%s on port %s with filepath %s", image, fileId, port, filepath)
-	resp.WriteHeader(200)
-	resp.Write([]byte(`{"success": true, "message": "Started webhook"}`))
-	return
-}
-
 // Checks if an image exists
 func imageCheckBuilder(images []string) error {
 	//log.Printf("[FIXME] ImageNames to check: %#v", images)
@@ -766,7 +594,7 @@ func imageCheckBuilder(images []string) error {
 }
 
 func hookTest() {
-	var hook Hook
+	var hook shuffle.Hook
 	err := json.Unmarshal([]byte(webhook), &hook)
 	log.Println(webhook)
 	if err != nil {
@@ -775,12 +603,12 @@ func hookTest() {
 	}
 
 	ctx := context.Background()
-	err = setHook(ctx, hook)
+	err = shuffle.SetHook(ctx, hook)
 	if err != nil {
 		log.Printf("Failed setting hook: %s", err)
 	}
 
-	returnHook, err := getHook(ctx, hook.Id)
+	returnHook, err := shuffle.GetHook(ctx, hook.Id)
 	if err != nil {
 		log.Printf("Failed getting hook %s (test): %s", hook.Id, err)
 	}
@@ -792,19 +620,19 @@ func hookTest() {
 
 //https://stackoverflow.com/questions/23935141/how-to-copy-docker-images-from-one-host-to-another-without-using-a-repository
 func getDockerImage(resp http.ResponseWriter, request *http.Request) {
-	cors := handleCors(resp, request)
+	cors := shuffle.HandleCors(resp, request)
 	if cors {
 		return
 	}
 
 	// Just here to verify that the user is logged in
-	_, err := handleApiAuthentication(resp, request)
+	//_, err := shuffle.HandleApiAuthentication(resp, request)
-	if err != nil {
+	//if err != nil {
-		log.Printf("Api authentication failed in validate swagger: %s", err)
+	//	log.Printf("[WARNING] Api authentication failed in DOWNLOAD IMAGE: %s", err)
-		resp.WriteHeader(401)
+	//	resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
+	//	resp.Write([]byte(`{"success": false}`))
-		return
+	//	return
-	}
+	//}
 
 	body, err := ioutil.ReadAll(request.Body)
 	if err != nil {
@@ -817,16 +645,6 @@ func getDockerImage(resp http.ResponseWriter, request *http.Request) {
 		Name string `datastore:"name" json:"name" yaml:"name"`
 	}
 
-	//body = []byte(`swagger: "2.0"`)
-	//body = []byte(`swagger: '1.0'`)
-	//newbody := string(body)
-	//newbody = strings.TrimSpace(newbody)
-	//body = []byte(newbody)
-	//log.Println(string(body))
-	//tmpbody, err := yaml.YAMLToJSON(body)
-	//log.Println(err)
-	//log.Println(string(tmpbody))
-
 	// This has to be done in a weird way because Datastore doesn't
 	// support map[string]interface and similar (openapi3.Swagger)
 	var version requestCheck
@@ -838,16 +656,10 @@ func getDockerImage(resp http.ResponseWriter, request *http.Request) {
 		return
 	}
 
-	log.Printf("Image to load: %s", version.Name)
+	//log.Printf("[DEBUG] Image to load: %s", version.Name)
-	//cli, err := client.NewEnvClient()
-	//if err != nil {
-	//	log.Println("Unable to create docker client")
-	//	return err
-	//}
-
 	dockercli, err := client.NewEnvClient()
 	if err != nil {
-		log.Printf("Unable to create docker client: %s", err)
+		log.Printf("[WARNING] Unable to create docker client: %s", err)
 		resp.WriteHeader(422)
 		resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed JSON marshalling: %s"}`, err)))
 		return
@@ -860,37 +672,241 @@ func getDockerImage(resp http.ResponseWriter, request *http.Request) {
 
 	img := types.ImageSummary{}
 	tagFound := ""
+
+	img2 := types.ImageSummary{}
+	tagFound2 := ""
+
+	alternativeNameSplit := strings.Split(version.Name, "/")
+	alternativeName := version.Name
+	if len(alternativeNameSplit) == 3 {
+		alternativeName = strings.Join(alternativeNameSplit[1:3], "/")
+	}
+
 	for _, image := range images {
 		for _, tag := range image.RepoTags {
-			log.Printf("Image: %s", tag)
-
 			if strings.ToLower(tag) == strings.ToLower(version.Name) {
 				img = image
 				tagFound = tag
 				break
 			}
+
+			if strings.ToLower(tag) == strings.ToLower(alternativeName) {
+				img2 = image
+				tagFound2 = tag
+			}
 		}
 	}
 
+	// REBUILDS THE APP
 	if len(img.ID) == 0 {
+		if len(img2.ID) == 0 {
+			workflowapps, err := shuffle.GetAllWorkflowApps(ctx, 0, 0)
+			//log.Printf("[INFO] Getting workflowapps for a rebuild. Got %d with err %#v", len(workflowapps), err)
+			if err == nil {
+				imageName := ""
+				imageVersion := ""
+				newNameSplit := strings.Split(version.Name, ":")
+				if len(newNameSplit) == 2 {
+					//log.Printf("[DEBUG] Found name %#v", newNameSplit)
+
+					findVersionSplit := strings.Split(newNameSplit[1], "_")
+					//log.Printf("[DEBUG] Found another split %#v", findVersionSplit)
+					if len(findVersionSplit) == 2 {
+						imageVersion = findVersionSplit[len(findVersionSplit)-1]
+						imageName = findVersionSplit[0]
+					} else if len(findVersionSplit) >= 2 {
+						imageVersion = findVersionSplit[len(findVersionSplit)-1]
+						imageName = strings.Join(findVersionSplit[0:len(findVersionSplit)-1], "_")
+					} else {
+						log.Printf("[DEBUG] Couldn't parse appname & version for %#v", findVersionSplit)
+					}
+				}
+
+				if len(imageName) > 0 && len(imageVersion) > 0 {
+					foundApp := shuffle.WorkflowApp{}
+					imageName = strings.ToLower(imageName)
+					imageVersion = strings.ToLower(imageVersion)
+					log.Printf("[DEBUG] Looking for appname %s with version %s", imageName, imageVersion)
+
+					for _, app := range workflowapps {
+						if strings.ToLower(strings.Replace(app.Name, " ", "_", -1)) == imageName && app.AppVersion == imageVersion {
+							if app.Generated {
+								log.Printf("[DEBUG] Found matching app %s:%s - %s", imageName, imageVersion, app.ID)
+								foundApp = app
+								break
+							} else {
+								log.Printf("[WARNING] Trying to rebuild app that isn't generated - not allowed. Looking further.")
+							}
+
+							//break
+						}
+					}
+
+					if len(foundApp.ID) > 0 {
+						openApiApp, err := shuffle.GetOpenApiDatastore(ctx, foundApp.ID)
+						if err != nil {
+							log.Printf("[ERROR] Failed getting OpenAPI app %s to database: %s", foundApp.ID, err)
+						} else {
+							log.Printf("[DEBUG] Found OpenAPI app for %s as generated - now building!", version.Name)
+							user := shuffle.User{}
+
+							//img = version.Name
+							if len(alternativeName) > 0 {
+								tagFound = alternativeName
+							} else {
+								tagFound = version.Name
+							}
+
+							buildSwaggerApp(resp, []byte(openApiApp.Body), user)
+						}
+					}
+				}
+			} else {
+				log.Printf("[WARNING] Couldn't find an image with registry name %s and %s", version.Name, alternativeName)
+				resp.WriteHeader(401)
+				resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "Couldn't find image %s"}`, version.Name)))
+				return
+			}
+		}
+
+		if len(tagFound) == 0 && len(tagFound2) > 0 {
+			img = img2
+			tagFound = tagFound2
+		}
+	}
+
+	//log.Printf("[INFO] Img found (%s): %#v", tagFound, img)
+	//log.Printf("[INFO] Img found to be downloaded by client: %s", tagFound)
+
+	newClient, err := newdockerclient.NewClientFromEnv()
+	if err != nil {
+		log.Printf("[ERROR] Failed setting up docker env: %#v", newClient)
 		resp.WriteHeader(401)
-		resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "Couldn't find image %s"}`, version.Name)))
+		resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "Couldn't make docker client"}`)))
 		return
 	}
-	_ = tagFound
 
-	/*
+	////https://github.com/fsouza/go-dockerclient/issues/600
-		log.Printf("IMg: %#v", img)
+	//defer fileReader.Close()
-		pullOptions := types.ImagePullOptions{}
+	opts := newdockerclient.ExportImageOptions{
-		log.Printf("[INFO] Pulling image %s", image)
+		Name:         tagFound,
-		reader, err := dockercli.ImagePull(ctx, tag, pullOptions)
+		OutputStream: resp,
-		if err != nil {
+	}
-			log.Printf("[ERROR] Failed getting image %s: %s", image, err)
+
+	if err := newClient.ExportImage(opts); err != nil {
+		log.Printf("[ERROR] FAILED to save image to file: %s", err)
+		resp.WriteHeader(401)
+		resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "Couldn't export image"}`)))
+		return
+	}
+
+	//resp.WriteHeader(200)
+}
+
+func activateWorkflowAppDocker(resp http.ResponseWriter, request *http.Request) {
+	cors := shuffle.HandleCors(resp, request)
+	if cors {
+		return
+	}
+
+	user, err := shuffle.HandleApiAuthentication(resp, request)
+	if err != nil {
+		log.Printf("[WARNING] Api authentication failed in get active apps: %s", err)
+		resp.WriteHeader(401)
+		resp.Write([]byte(`{"success": false}`))
+		return
+	}
+
+	if user.Role == "org-reader" {
+		log.Printf("[WARNING] Org-reader doesn't have access to activate workflow app (shared): %s (%s)", user.Username, user.Id)
+		resp.WriteHeader(401)
+		resp.Write([]byte(`{"success": false, "reason": "Read only user"}`))
+		return
+	}
+
+	ctx := context.Background()
+	location := strings.Split(request.URL.String(), "/")
+	var fileId string
+	if location[1] == "api" {
+		if len(location) <= 4 {
+			resp.WriteHeader(401)
+			resp.Write([]byte(`{"success": false}`))
+			return
 		}
 
-		io.Copy(os.Stdout, r)
+		fileId = location[4]
-	*/
+	}
+
+	app, err := shuffle.GetApp(ctx, fileId, user, false)
+	if err != nil {
+		appName := request.URL.Query().Get("app_name")
+		appVersion := request.URL.Query().Get("app_version")
+
+		if len(appName) > 0 && len(appVersion) > 0 {
+			apps, err := shuffle.FindWorkflowAppByName(ctx, appName)
+			//log.Printf("[INFO] Found %d apps for %s", len(apps), appName)
+			if err != nil || len(apps) == 0 {
+				log.Printf("[WARNING] Error getting app %s (app config): %s", appName, err)
+				resp.WriteHeader(401)
+				resp.Write([]byte(`{"success": false, "reason": "App doesn't exist"}`))
+				return
+			}
+
+			selectedApp := shuffle.WorkflowApp{}
+			for _, app := range apps {
+				if !app.Sharing && !app.Public {
+					continue
+				}
+
+				if app.Name == appName {
+					selectedApp = app
+				}
+
+				if app.Name == appName && app.AppVersion == appVersion {
+					selectedApp = app
+				}
+			}
+
+			app = &selectedApp
+		} else {
+			log.Printf("[WARNING] Error getting app with ID %s (app config): %s", fileId, err)
+			resp.WriteHeader(401)
+			resp.Write([]byte(`{"success": false, "reason": "App doesn't exist"}`))
+			return
+		}
+	}
+
+	if app.Sharing || app.Public {
+		org, err := shuffle.GetOrg(ctx, user.ActiveOrg.Id)
+		if err == nil {
+			added := false
+			if !shuffle.ArrayContains(org.ActiveApps, app.ID) {
+				org.ActiveApps = append(org.ActiveApps, app.ID)
+				added = true
+			}
+
+			if added {
+				err = shuffle.SetOrg(ctx, *org, org.Id)
+				if err != nil {
+					log.Printf("[WARNING] Failed setting org when autoadding apps on save: %s", err)
+				} else {
+					log.Printf("[INFO] Added public app %s (%s) to org %s (%s)", app.Name, app.ID, user.ActiveOrg.Name, user.ActiveOrg.Id)
+					cacheKey := fmt.Sprintf("apps_%s", user.Id)
+					shuffle.DeleteCache(ctx, cacheKey)
+				}
+			}
+		}
+	} else {
+		log.Printf("[WARNING] User is trying to activate %s which is NOT public", app.Name)
+		resp.WriteHeader(401)
+		resp.Write([]byte(`{"success": false}`))
+		return
+	}
+
+	log.Printf("[DEBUG] App %s (%s) activated for org %s by user %s", app.Name, app.ID, user.ActiveOrg.Id, user.Username)
+
+	// If onprem, it should autobuild the container(s) from here
 
 	resp.WriteHeader(200)
-	resp.Write([]byte(fmt.Sprintf(`{"success": true, "message": "Downloading image %s"}`, version.Name)))
+	resp.Write([]byte(`{"success": true}`))
 }

backend/go-app/files.go

@@ -1,873 +0,0 @@
-package main
-
-/*
-	Handles files within Workflows.of Shuffle
-*/
-
-import (
-	"bytes"
-	"context"
-	"crypto/sha256"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"io"
-	"io/ioutil"
-	"log"
-	"net/http"
-	"os"
-	"strconv"
-	"strings"
-	"time"
-
-	"cloud.google.com/go/datastore"
-	"github.com/satori/go.uuid"
-)
-
-type File struct {
-	Id           string   `json:"id" datastore:"id"`
-	Type         string   `json:"type" datastore:"type"`
-	CreatedAt    int64    `json:"created_at" datastore:"created_at"`
-	UpdatedAt    int64    `json:"updated_at" datastore:"updated_at"`
-	MetaAccessAt int64    `json:"meta_access_at" datastore:"meta_access_at"`
-	DownloadAt   int64    `json:"last_downloaded" datastore:"last_downloaded"`
-	Description  string   `json:"description" datastore:"description"`
-	ExpiresAt    string   `json:"expires_at" datastore:"expires_at"`
-	Status       string   `json:"status" datastore:"status"`
-	Filename     string   `json:"filename" datastore:"filename"`
-	URL          string   `json:"url" datastore:"org"`
-	OrgId        string   `json:"org_id" datastore:"org_id"`
-	WorkflowId   string   `json:"workflow_id" datastore:"workflow_id"`
-	Workflows    []string `json:"workflows" datastore:"workflows"`
-	DownloadPath string   `json:"download_path" datastore:"download_path"`
-	Md5sum       string   `json:"md5_sum" datastore:"md5_sum"`
-	Sha256sum    string   `json:"sha256_sum" datastore:"sha256_sum"`
-	FileSize     int64    `json:"filesize" datastore:"filesize"`
-	Duplicate    bool     `json:"duplicate" datastore:"duplicate"`
-	Subflows     []string `json:"subflows" datastore:"subflows"`
-}
-
-var basepath = os.Getenv("SHUFFLE_FILE_LOCATION")
-
-func fileAuthentication(request *http.Request) (string, error) {
-	executionId, ok := request.URL.Query()["execution_id"]
-	if ok && len(executionId) > 0 {
-		ctx := context.Background()
-		workflowExecution, err := getWorkflowExecution(ctx, executionId[0])
-		if err != nil {
-			log.Printf("[ERROR] Couldn't find execution ID %s", executionId[0])
-			return "", err
-		}
-
-		apikey := request.Header.Get("Authorization")
-		if !strings.HasPrefix(apikey, "Bearer ") {
-			log.Printf("[ERROR} Apikey doesn't start with bearer (2)")
-			return "", errors.New("No auth key found")
-		}
-
-		apikeyCheck := strings.Split(apikey, " ")
-		if len(apikeyCheck) != 2 {
-			log.Printf("[ERROR] Invalid format for apikey (2)")
-			return "", errors.New("No space in authkey")
-		}
-
-		// This is annoying af and is done because of maxlength lol
-		newApikey := apikeyCheck[1]
-		if newApikey != workflowExecution.Authorization {
-			//log.Printf("[ERROR] Bad apikey for execution %s. %s vs %s", executionId[0], apikey, workflowExecution.Authorization)
-			log.Printf("[ERROR] Bad apikey for execution %s.", executionId[0])
-			//%s vs %s", executionId[0], apikey, workflowExecution.Authorization)
-			return "", errors.New("Bad authorization key")
-		}
-
-		log.Printf("[INFO] Authorization is correct for execution %s!", executionId[0])
-		//%s vs %s. Setting Org", executionId, apikey, workflowExecution.Authorization)
-		if len(workflowExecution.ExecutionOrg) > 0 {
-			return workflowExecution.ExecutionOrg, nil
-		} else if len(workflowExecution.Workflow.ExecutingOrg.Id) > 0 {
-			return workflowExecution.ExecutionOrg, nil
-		} else {
-			log.Printf("[ERROR] Couldn't find org for workflow execution, but auth was correct.")
-		}
-	}
-
-	return "", errors.New("No execution id specified")
-}
-
-// https://golangcode.com/check-if-a-file-exists/
-func fileExists(filename string) bool {
-	info, err := os.Stat(filename)
-	if os.IsNotExist(err) {
-		return false
-	}
-	return !info.IsDir()
-}
-
-func handleGetFiles(resp http.ResponseWriter, request *http.Request) {
-	cors := handleCors(resp, request)
-	if cors {
-		return
-	}
-
-	// 1. Check user directly
-	// 2. Check workflow execution authorization
-	user, err := handleApiAuthentication(resp, request)
-	if err != nil {
-		log.Printf("[INFO] INITIAL Api authentication failed in file LIST: %s", err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	if user.Role != "admin" {
-		log.Printf("[AUTH] User isn't admin")
-		resp.WriteHeader(401)
-		resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Need to be admin"}`)))
-		return
-	}
-
-	ctx := context.Background()
-	files, err := getAllFiles(ctx, user.ActiveOrg.Id)
-	if err != nil {
-		log.Printf("[ERROR] Failed to get files: %s", err)
-		resp.WriteHeader(500)
-		resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Error getting files."}`)))
-		return
-	}
-
-	log.Printf("Got %d files for org %s", len(files), user.ActiveOrg.Id)
-	newBody, err := json.Marshal(files)
-	if err != nil {
-		log.Printf("[ERROR] Failed marshaling files: %s", err)
-		resp.WriteHeader(500)
-		resp.Write([]byte(`{"success": false, "reason": "Failed to marshal files"}`))
-		return
-	}
-
-	resp.WriteHeader(200)
-	resp.Write([]byte(newBody))
-}
-
-func handleGetFileMeta(resp http.ResponseWriter, request *http.Request) {
-	cors := handleCors(resp, request)
-	if cors {
-		return
-	}
-
-	var fileId string
-	location := strings.Split(request.URL.String(), "/")
-	if location[1] == "api" {
-		if len(location) <= 4 {
-			log.Printf("[INFO] Path too short: %d", len(location))
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-
-		fileId = location[4]
-	}
-
-	if strings.Contains(fileId, "?") {
-		fileId = strings.Split(fileId, "?")[0]
-	}
-
-	if len(fileId) != 36 {
-		log.Printf("Bad format for fileId %s", fileId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
-		return
-	}
-
-	log.Printf("\n\n[INFO] User is trying to GET File Meta for %s\n\n", fileId)
-
-	// 1. Check user directly
-	// 2. Check workflow execution authorization
-	user, err := handleApiAuthentication(resp, request)
-	if err != nil {
-		log.Printf("[INFO] INITIAL Api authentication failed in file deletion: %s", err)
-
-		orgId, err := fileAuthentication(request)
-		if err != nil {
-			log.Printf("[ERROR] Bad file authentication in get: %s", err)
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-
-		user.ActiveOrg.Id = orgId
-		user.Username = "Execution File API"
-	}
-
-	// 1. Verify if the user has access to the file: org_id and workflow
-	log.Printf("[INFO] Should GET FILE META for %s if user has access", fileId)
-	ctx := context.Background()
-	file, err := getFile(ctx, fileId)
-	if err != nil {
-		log.Printf("[INFO] File %s not found: %s", fileId, err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	found := false
-	if file.OrgId == user.ActiveOrg.Id {
-		found = true
-	} else {
-		for _, item := range user.Orgs {
-			if item == file.OrgId {
-				found = true
-				break
-			}
-		}
-	}
-
-	if !found {
-		log.Printf("[INFO] User %s doesn't have access to %s", user.Username, fileId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	newBody, err := json.Marshal(file)
-	if err != nil {
-		resp.WriteHeader(500)
-		resp.Write([]byte(`{"success": false, "reason": "Failed to marshal filedata"}`))
-		return
-	}
-
-	log.Printf("[INFO] Successfully got file meta for %s", fileId)
-	resp.WriteHeader(200)
-	resp.Write([]byte(newBody))
-}
-
-func handleDeleteFile(resp http.ResponseWriter, request *http.Request) {
-	cors := handleCors(resp, request)
-	if cors {
-		return
-	}
-
-	var fileId string
-	location := strings.Split(request.URL.String(), "/")
-	if location[1] == "api" {
-		if len(location) <= 4 {
-			log.Printf("[INFO] Path too short: %d", len(location))
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-
-		fileId = location[4]
-	}
-
-	if strings.Contains(fileId, "?") {
-		fileId = strings.Split(fileId, "?")[0]
-	}
-
-	if len(fileId) != 36 {
-		log.Printf("Bad format for fileId %s", fileId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
-		return
-	}
-
-	log.Printf("\n\n[INFO] User is trying to delete file %s\n\n", fileId)
-
-	// 1. Check user directly
-	// 2. Check workflow execution authorization
-	user, err := handleApiAuthentication(resp, request)
-	if err != nil {
-		log.Printf("[INFO] INITIAL Api authentication failed in file deletion: %s", err)
-
-		orgId, err := fileAuthentication(request)
-		if err != nil {
-			log.Printf("[ERROR] Bad file authentication in get: %s", err)
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-
-		user.ActiveOrg.Id = orgId
-		user.Username = "Execution File API"
-	}
-
-	// 1. Verify if the user has access to the file: org_id and workflow
-	log.Printf("[INFO] Should DELETE file %s if user has access", fileId)
-	ctx := context.Background()
-	file, err := getFile(ctx, fileId)
-	if err != nil {
-		log.Printf("[INFO] File %s not found: %s", fileId, err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	found := false
-	if file.OrgId == user.ActiveOrg.Id {
-		found = true
-	} else {
-		for _, item := range user.Orgs {
-			if item == file.OrgId {
-				found = true
-				break
-			}
-		}
-	}
-
-	if !found {
-		log.Printf("[INFO] User %s doesn't have access to %s", user.Username, fileId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	if file.Status == "deleted" {
-		log.Printf("[INFO] File with ID %s is already deleted.", fileId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	if fileExists(file.DownloadPath) {
-		err = os.Remove(file.DownloadPath)
-		if err != nil {
-			log.Printf("[ERROR] Failed deleting file locally: %s", err)
-			resp.WriteHeader(401)
-			resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed deleting filein path %s"}`, file.DownloadPath)))
-			return
-		}
-
-		log.Printf("[INFO] Deleted file %s locally. Next is database.", file.DownloadPath)
-	} else {
-		log.Printf("[ERROR] File doesn't exist. Can't delete. Should maybe delete file anyway?")
-		resp.WriteHeader(200)
-		resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "File in location %s doesn't exist"}`, file.DownloadPath)))
-		return
-	}
-
-	file.Status = "deleted"
-	err = setFile(ctx, *file)
-	if err != nil {
-		log.Printf("[ERROR] Failed setting file to deleted")
-		resp.WriteHeader(500)
-		resp.Write([]byte(`{"success": false, "reason": "Failed setting file to deleted"}`))
-		return
-	}
-
-	/*
-		//Actually delete it?
-		err = DeleteKey(ctx, "files", fileId)
-		if err != nil {
-			log.Printf("Failed deleting file with ID %s: %s", fileId, err)
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-	*/
-
-	log.Printf("[INFO] Successfully deleted file %s", fileId)
-	resp.WriteHeader(200)
-	resp.Write([]byte(`{"success": true}`))
-}
-
-func handleGetFileContent(resp http.ResponseWriter, request *http.Request) {
-	cors := handleCors(resp, request)
-	if cors {
-		return
-	}
-
-	var fileId string
-	location := strings.Split(request.URL.String(), "/")
-	if location[1] == "api" {
-		if len(location) <= 4 {
-			log.Printf("Path too short: %d", len(location))
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-
-		fileId = location[4]
-	}
-
-	if len(fileId) != 36 {
-		log.Printf("Bad format for fileId %s", fileId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
-		return
-	}
-
-	log.Printf("\n\n[INFO] User is trying to download file %s\n\n", fileId)
-
-	// 1. Check user directly
-	// 2. Check workflow execution authorization
-	user, err := handleApiAuthentication(resp, request)
-	if err != nil {
-		log.Printf("INITIAL Api authentication failed in file download: %s", err)
-
-		orgId, err := fileAuthentication(request)
-		if err != nil {
-			log.Printf("Bad file authentication in get: %s", err)
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-
-		user.ActiveOrg.Id = orgId
-		user.Username = "Execution File API"
-		/*
-			} else {
-				resp.WriteHeader(401)
-				resp.Write([]byte(`{"success": false}`))
-				return
-			}
-		*/
-	}
-
-	// 1. Verify if the user has access to the file: org_id and workflow
-	log.Printf("[INFO] Should get file %s", fileId)
-	ctx := context.Background()
-	file, err := getFile(ctx, fileId)
-	if err != nil {
-		log.Printf("[ERROR] File %s not found: %s", fileId, err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	found := false
-	if file.OrgId == user.ActiveOrg.Id {
-		found = true
-	} else {
-		for _, item := range user.Orgs {
-			if item == file.OrgId {
-				found = true
-				break
-			}
-		}
-	}
-
-	if !found {
-		log.Printf("User %s doesn't have access to %s", user.Username, fileId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	if file.Status != "active" {
-		log.Printf("[ERROR] File status isn't active, but %s. Can't continue.", file.Status)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "The file isn't ready to be downloaded yet. Status required: active"}`))
-		return
-	}
-
-	// Fixme: More auth: org and workflow!
-	downloadPath := file.DownloadPath
-	log.Printf("[INFO] Downloadpath: %s", downloadPath)
-	Openfile, err := os.Open(downloadPath)
-	defer Openfile.Close() //Close after function return
-	if err != nil {
-		file.Status = "deleted"
-		err = setFile(ctx, *file)
-		if err != nil {
-			log.Printf("Failed setting file to uploading")
-			resp.WriteHeader(500)
-			resp.Write([]byte(`{"success": false, "reason": "Failed setting file to uploading"}`))
-			return
-		}
-
-		//File not found, send 404
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "File doesn't exist locally"}`))
-		return
-	}
-
-	//File is found, create and send the correct headers
-	//Get the Content-Type of the file
-	//Create a buffer to store the header of the file in
-	FileHeader := make([]byte, 512)
-	//Copy the headers into the FileHeader buffer
-	Openfile.Read(FileHeader)
-	//Get content type of file
-	FileContentType := http.DetectContentType(FileHeader)
-
-	//Get the file size
-	FileStat, _ := Openfile.Stat()                     //Get info from file
-	FileSize := strconv.FormatInt(FileStat.Size(), 10) //Get file size as a string
-
-	//Send the headers
-	resp.Header().Set("Content-Disposition", "attachment; filename="+fileId)
-	resp.Header().Set("Content-Type", FileContentType)
-	resp.Header().Set("Content-Length", FileSize)
-
-	//Send the file
-	//We read 512 bytes from the file already, so we reset the offset back to 0
-	Openfile.Seek(0, 0)
-	io.Copy(resp, Openfile) //'Copy' the file to the client
-	return
-
-	//log.Printf("Should download file %s", downloadPath)
-}
-func handleUploadFile(resp http.ResponseWriter, request *http.Request) {
-	cors := handleCors(resp, request)
-	if cors {
-		return
-	}
-
-	var fileId string
-	location := strings.Split(request.URL.String(), "/")
-	if location[1] == "api" {
-		if len(location) <= 4 {
-			log.Printf("Path too short: %d", len(location))
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-
-		fileId = location[4]
-	}
-
-	if len(fileId) != 36 {
-		log.Printf("Bad format for fileId %s", fileId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
-		return
-	}
-
-	// 1. Check user directly
-	// 2. Check workflow execution authorization
-	user, err := handleApiAuthentication(resp, request)
-	if err != nil {
-		log.Printf("INITIAL Api authentication failed in file upload: %s", err)
-
-		orgId, err := fileAuthentication(request)
-		if err != nil {
-			log.Printf("Bad file authentication in create file: %s", err)
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-
-		user.ActiveOrg.Id = orgId
-		user.Username = "Execution File API"
-	}
-
-	log.Printf("[INFO] Should UPLOAD file %s if user has access", fileId)
-	ctx := context.Background()
-	file, err := getFile(ctx, fileId)
-	if err != nil {
-		log.Printf("File %s not found: %s", fileId, err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	found := false
-	if file.OrgId == user.ActiveOrg.Id {
-		found = true
-	} else {
-		for _, item := range user.Orgs {
-			if item == file.OrgId {
-				found = true
-				break
-			}
-		}
-	}
-
-	if !found {
-		log.Printf("User %s doesn't have access to %s", user.Username, fileId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	log.Printf("[INFO] STATUS: %s", file.Status)
-	if file.Status != "created" {
-		log.Printf("File status isn't created. Can't upload.")
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "This file already has data."}`))
-		return
-	}
-
-	request.ParseMultipartForm(32 << 20)
-	parsedFile, _, err := request.FormFile("shuffle_file")
-	if err != nil {
-		log.Printf("[ERROR] Couldn't upload file: %s", err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Failed uploading file"}`))
-		return
-	}
-	defer parsedFile.Close()
-
-	file.Status = "uploading"
-	err = setFile(ctx, *file)
-	if err != nil {
-		log.Printf("Failed setting file to uploading")
-		resp.WriteHeader(500)
-		resp.Write([]byte(`{"success": false, "reason": "Failed setting file to uploading"}`))
-		return
-	}
-
-	// Can be used for validation files for change
-	var buf bytes.Buffer
-	io.Copy(&buf, parsedFile)
-	contents := buf.Bytes()
-	file.FileSize = int64(len(contents))
-	md5 := md5sum(contents)
-	buf.Reset()
-
-	sha256Sum := sha256.Sum256(contents)
-	//parsedFile.Reset()
-
-	f, err := os.OpenFile(file.DownloadPath, os.O_WRONLY|os.O_CREATE, os.ModePerm)
-	if err != nil {
-		// Rolling back file
-		file.Status = "created"
-		setFile(ctx, *file)
-
-		log.Printf("[ERROR] Failed uploading and creating file: %s", err)
-		resp.WriteHeader(500)
-		resp.Write([]byte(`{"success": false}`))
-		return
-	}
-
-	defer f.Close()
-	parsedFile.Seek(0, io.SeekStart)
-	io.Copy(f, parsedFile)
-
-	// FIXME: Set this one to 200 anyway? Can't download file then tho..
-	file.Status = "active"
-	file.Md5sum = md5
-	file.Sha256sum = fmt.Sprintf("%x", sha256Sum)
-	log.Printf("[INFO] MD5 for file %s (%s) is %s and SHA256 is %s", file.Filename, file.Id, file.Md5sum, file.Sha256sum)
-
-	err = setFile(ctx, *file)
-	if err != nil {
-		log.Printf("[ERROR] Failed setting file back to active")
-		resp.WriteHeader(500)
-		resp.Write([]byte(`{"success": false, "reason": "Failed setting file to active"}`))
-		return
-	}
-
-	log.Printf("[INFO] Successfully uploaded file ID %s", file.Id)
-	resp.WriteHeader(200)
-	resp.Write([]byte(fmt.Sprintf(`{"success": true}`)))
-}
-
-func handleCreateFile(resp http.ResponseWriter, request *http.Request) {
-	cors := handleCors(resp, request)
-	if cors {
-		return
-	}
-
-	// 1. Check user directly
-	// 2. Check workflow execution authorization
-	user, err := handleApiAuthentication(resp, request)
-	if err != nil {
-		log.Printf("[INFO] INITIAL Api authentication failed in file creation: %s", err)
-
-		orgId, err := fileAuthentication(request)
-		if err != nil {
-			log.Printf("[ERROR] Bad file authentication in create file: %s", err)
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false}`))
-			return
-		}
-
-		user.ActiveOrg.Id = orgId
-		user.Username = "Execution File API"
-	}
-
-	body, err := ioutil.ReadAll(request.Body)
-	if err != nil {
-		log.Println("Failed reading body")
-		resp.WriteHeader(401)
-		resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed to read data"}`)))
-		return
-	}
-
-	type FileStructure struct {
-		Filename   string `json:"filename"`
-		OrgId      string `json:"org_id"`
-		WorkflowId string `json:"workflow_id"`
-	}
-
-	var curfile FileStructure
-	err = json.Unmarshal(body, &curfile)
-	if err != nil {
-		log.Printf("[ERROR] Failed unmarshaling: %s", err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed to unmarshal data"}`)))
-		return
-	}
-
-	// Loads of validation below
-	if len(curfile.Filename) == 0 || len(curfile.OrgId) == 0 || len(curfile.WorkflowId) == 0 {
-		log.Printf("[ERROR] Missing field during fileupload. Required: filename, org_id, workflow_id")
-		log.Printf("INPUT: %s", string(body))
-		resp.WriteHeader(401)
-		resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Missing field. Required: filename, org_id, workflow_id"}`)))
-		return
-	}
-
-	ctx := context.Background()
-	if user.ActiveOrg.Id != curfile.OrgId {
-		log.Printf("[ERROR] User can't access org %s", curfile.OrgId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Error with organization"}`))
-		return
-	}
-
-	// Try to get the org and workflow in case they don't exist
-	workflow, err := getWorkflow(ctx, curfile.WorkflowId)
-	if err != nil {
-		log.Printf("[ERROR] Workflow %s doesn't exist.", curfile.WorkflowId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Error with workflow id or org id"}`))
-		return
-	}
-
-	_, err = getOrg(ctx, curfile.OrgId)
-	if err != nil {
-		log.Printf("[ERROR] Org %s doesn't exist.", curfile.OrgId)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Error with workflow id or org id"}`))
-		return
-	}
-
-	if workflow.ExecutingOrg.Id != curfile.OrgId {
-		found := false
-		for _, curorg := range workflow.Org {
-			if curorg.Id == curfile.OrgId {
-				found = true
-				break
-			}
-		}
-
-		if !found {
-			log.Printf("[ERROR] Org %s doesn't have access to %s.", curfile.OrgId, curfile.WorkflowId)
-			resp.WriteHeader(401)
-			resp.Write([]byte(`{"success": false, "reason": "Error with workflow id or org id"}`))
-			return
-		}
-	}
-
-	if strings.Contains(curfile.Filename, "/") || strings.Contains(curfile.Filename, `"`) || strings.Contains(curfile.Filename, "..") || strings.Contains(curfile.Filename, "~") {
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Invalid characters in filename"}`))
-		return
-	}
-
-	// 1. Create the file object.
-	if len(basepath) == 0 {
-		basepath = "shuffle-files"
-	}
-	folderPath := fmt.Sprintf("%s/%s/%s", basepath, curfile.OrgId, curfile.WorkflowId)
-
-	// Try to make the full file location
-	err = os.MkdirAll(folderPath, os.ModePerm)
-	if err != nil {
-		log.Printf("[ERROR] Writing issue for file location creation: %s", err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Failed creating upload location"}`))
-		return
-	}
-
-	filename := curfile.Filename
-	fileId := uuid.NewV4().String()
-	downloadPath := fmt.Sprintf("%s/%s", folderPath, fileId)
-
-	duplicateWorkflows := []string{}
-	for _, trigger := range workflow.Triggers {
-		if trigger.AppName == "Shuffle Workflow" && trigger.TriggerType == "SUBFLOW" {
-			for _, parameter := range trigger.Parameters {
-				if parameter.Name == "workflow" && len(parameter.Value) > 0 {
-
-					found := false
-					for _, workflow := range duplicateWorkflows {
-						if workflow == parameter.Value {
-							found = true
-							break
-						}
-					}
-
-					if !found {
-						duplicateWorkflows = append(duplicateWorkflows, parameter.Value)
-					}
-
-					break
-				}
-			}
-		}
-	}
-
-	timeNow := time.Now().Unix()
-	newFile := File{
-		Id:           fileId,
-		CreatedAt:    timeNow,
-		UpdatedAt:    timeNow,
-		Description:  "",
-		Status:       "created",
-		Filename:     filename,
-		OrgId:        curfile.OrgId,
-		WorkflowId:   curfile.WorkflowId,
-		DownloadPath: downloadPath,
-		Subflows:     duplicateWorkflows,
-	}
-
-	err = setFile(ctx, newFile)
-	if err != nil {
-		log.Printf("[ERROR] Failed setting file: %s", err)
-		resp.WriteHeader(401)
-		resp.Write([]byte(`{"success": false, "reason": "Failed setting file reference"}`))
-		return
-	} else {
-		log.Printf("[INFO] Created file %s", newFile.DownloadPath)
-	}
-
-	resp.WriteHeader(200)
-	resp.Write([]byte(fmt.Sprintf(`{"success": true, "id": "%s"}`, fileId)))
-
-}
-
-func getFile(ctx context.Context, id string) (*File, error) {
-	key := datastore.NameKey("Files", id, nil)
-	curFile := &File{}
-	if err := dbclient.Get(ctx, key, curFile); err != nil {
-		return &File{}, err
-	}
-
-	return curFile, nil
-}
-
-func setFile(ctx context.Context, file File) error {
-	// clear session_token and API_token for user
-	timeNow := time.Now().Unix()
-	file.UpdatedAt = timeNow
-
-	k := datastore.NameKey("Files", file.Id, nil)
-	if _, err := dbclient.Put(ctx, k, &file); err != nil {
-		log.Println(err)
-		return err
-	}
-
-	return nil
-}
-
-func getAllFiles(ctx context.Context, orgId string) ([]File, error) {
-	var files []File
-	q := datastore.NewQuery("Files").Filter("org_id =", orgId).Order("-updated_at").Limit(100)
-
-	_, err := dbclient.GetAll(ctx, q, &files)
-	if err != nil {
-		if strings.Contains(fmt.Sprintf("%s", err), "ResourceExhausted") {
-			q = q.Limit(50)
-			_, err := dbclient.GetAll(ctx, q, &files)
-			if err != nil {
-				return []File{}, err
-			}
-		} else {
-			return []File{}, err
-		}
-	}
-
-	return files, nil
-}

backend/go-app/go.mod

@@ -1,37 +1,35 @@
-module shuffle
+module main
 
-go 1.13
+go 1.16
+
+replace github.com/shuffle/shuffle-shared => ../../../shuffle-shared
+
+//replace github.com/frikky/kin-openapi => ../../../../git/kin-openapi
+//replace github.com/frikky/go-elasticsearch => ../../../../git/go-elasticsearch
 
 require (
-	cloud.google.com/go v0.57.0
+	cloud.google.com/go/datastore v1.6.0
-	cloud.google.com/go/datastore v1.1.0
+	cloud.google.com/go/iam v0.1.1 // indirect
-	cloud.google.com/go/pubsub v1.3.1
+	cloud.google.com/go/pubsub v1.17.1
-	cloud.google.com/go/storage v1.7.0
+	cloud.google.com/go/storage v1.18.2
-	github.com/Microsoft/go-winio v0.4.14 // indirect
 	github.com/basgys/goxml2json v1.1.0
 	github.com/carlescere/scheduler v0.0.0-20170109141437-ee74d2f83d82
-	github.com/docker/distribution v2.7.1+incompatible // indirect
+	github.com/docker/docker v20.10.12+incompatible
-	github.com/docker/docker v1.13.1
+	github.com/frikky/kin-openapi v0.41.0
-	github.com/docker/go-connections v0.4.0
+	github.com/fsouza/go-dockerclient v1.7.7
-	github.com/docker/go-units v0.4.0 // indirect
-	github.com/getkin/kin-openapi v0.8.0
 	github.com/ghodss/yaml v1.0.0
-	github.com/go-git/go-billy/v5 v5.0.0
+	github.com/go-git/go-billy/v5 v5.3.1
-	github.com/go-git/go-git/v5 v5.0.0
+	github.com/go-git/go-git/v5 v5.4.2
-	github.com/google/go-github/v28 v28.1.1
+	github.com/gorilla/mux v1.8.0
-	github.com/gorilla/handlers v1.4.2 // indirect
+	github.com/h2non/filetype v1.1.3
-	github.com/gorilla/mux v1.7.4
+	github.com/nirasan/go-oauth-pkce-code-verifier v0.0.0-20170819232839-0fbfe93532da // indirect
-	github.com/h2non/filetype v1.0.12
-	github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
-	github.com/patrickmn/go-cache v2.1.0+incompatible
 	github.com/satori/go.uuid v1.2.0
-	golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79
+	github.com/shuffle/shuffle-shared v0.2.29
-	golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
+	go4.org v0.0.0-20201209231011-d4a079459e60 // indirect
-	google.golang.org/api v0.23.0
+	golang.org/x/crypto v0.0.0-20220112180741-5e0467b6c7ce
-	google.golang.org/appengine v1.6.6
+	google.golang.org/api v0.65.0
-	google.golang.org/genproto v0.0.0-20200507105951-43844f6eee31
+	google.golang.org/appengine v1.6.7
-	google.golang.org/grpc v1.29.1
+	google.golang.org/grpc v1.43.0
 	gopkg.in/src-d/go-git.v4 v4.13.1
-	gopkg.in/yaml.v2 v2.2.8
+	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
-	gopkg.in/yaml.v3 v3.0.0-20200506231410-2ff61e1afc86
 )

backend/go-app/go.sum

@@ -1,435 +0,0 @@
-cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
-cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
-cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
-cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
-cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
-cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
-cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
-cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
-cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=
-cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
-cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=
-cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
-cloud.google.com/go v0.57.0 h1:EpMNVUorLiZIELdMZbCYX/ByTFCdoYopYAGxaGVz9ms=
-cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
-cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
-cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
-cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
-cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
-cloud.google.com/go/bigquery v1.6.0/go.mod h1:hyFDG0qSGdHNz8Q6nDN8rYIkld0q/+5uBZaelxiDLfE=
-cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
-cloud.google.com/go/datastore v1.1.0 h1:/May9ojXjRkPBNVrq+oWLqmWCkr4OU5uRY29bu0mRyQ=
-cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
-cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
-cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
-cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
-cloud.google.com/go/pubsub v1.3.1 h1:ukjixP1wl0LpnZ6LWtZJ0mX5tBmjp1f8Sqer8Z2OMUU=
-cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=
-cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
-cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
-cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
-cloud.google.com/go/storage v1.7.0 h1:DzdLPI8Em+DEk7IzA2a10ivq3mxIEASC9GeNJ6FFt5Q=
-cloud.google.com/go/storage v1.7.0/go.mod h1:jGMIBwF+L/tL6WN/W5InNgYYu4HP0DvGB6rQ1mufWfs=
-dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
-github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
-github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
-github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
-github.com/Microsoft/go-winio v0.4.14 h1:+hMXMk01us9KgxGb7ftKQt2Xpf5hH/yky+TDA+qxleU=
-github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA=
-github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7/go.mod h1:6zEj6s6u/ghQa61ZWa/C2Aw3RkjiTBOix7dkqa1VLIs=
-github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c=
-github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
-github.com/basgys/goxml2json v1.1.0 h1:4ln5i4rseYfXNd86lGEB+Vi652IsIXIvggKM/BhUKVw=
-github.com/basgys/goxml2json v1.1.0/go.mod h1:wH7a5Np/Q4QoECFIU8zTQlZwZkrilY0itPfecMw41Dw=
-github.com/carlescere/scheduler v0.0.0-20170109141437-ee74d2f83d82 h1:9bAydALqAjBfPHd/eAiJBHnMZUYov8m2PkXVr+YGQeI=
-github.com/carlescere/scheduler v0.0.0-20170109141437-ee74d2f83d82/go.mod h1:tyA14J0sA3Hph4dt+AfCjPrYR13+vVodshQSM7km9qw=
-github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
-github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
-github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
-github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
-github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
-github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
-github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
-github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
-github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug=
-github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
-github.com/docker/docker v1.13.1 h1:IkZjBSIc8hBjLpqeAbeE5mca5mNgeatLHBy3GO78BWo=
-github.com/docker/docker v1.13.1/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
-github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
-github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
-github.com/docker/go-units v0.4.0 h1:3uh0PgVws3nIA0Q+MwDC8yjEPf9zjRfZZWXZYDct3Tw=
-github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
-github.com/emirpasic/gods v1.12.0 h1:QAUIPSaCu4G+POclxeqb3F+WPpdKqFGlw36+yOzGlrg=
-github.com/emirpasic/gods v1.12.0/go.mod h1:YfzfFFoVP/catgzJb4IKIqXjX78Ha8FMSDh3ymbK86o=
-github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
-github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
-github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
-github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
-github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
-github.com/getkin/kin-openapi v0.8.0 h1:a6TQjTqwkyscC4/hShJX7WhCVE+4bi9lzw61XHQW5hE=
-github.com/getkin/kin-openapi v0.8.0/go.mod h1:zZQMFkVgRHCdhgb6ihCTIo9dyDZFvX0k/xAKqw1FhPw=
-github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
-github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
-github.com/gliderlabs/ssh v0.2.2/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0=
-github.com/go-git/gcfg v1.5.0 h1:Q5ViNfGF8zFgyJWPqYwA7qGFoMTEiBmdlkcfRmpIMa4=
-github.com/go-git/gcfg v1.5.0/go.mod h1:5m20vg6GwYabIxaOonVkTdrILxQMpEShl1xiMF4ua+E=
-github.com/go-git/go-billy v4.2.0+incompatible h1:Z6QtVXd5tjxUtcODLugkJg4WaZnGg13CD8qB9pr+7q0=
-github.com/go-git/go-billy/v5 v5.0.0 h1:7NQHvd9FVid8VL4qVUMm8XifBK+2xCoZ2lSk0agRrHM=
-github.com/go-git/go-billy/v5 v5.0.0/go.mod h1:pmpqyWchKfYfrkb/UVH4otLvyi/5gJlGI4Hb3ZqZ3W0=
-github.com/go-git/go-git v4.7.0+incompatible h1:+W9rgGY4DOKKdX2x6HxSR7HNeTxqiKrOvKnuittYVdA=
-github.com/go-git/go-git-fixtures/v4 v4.0.1/go.mod h1:m+ICp2rF3jDhFgEZ/8yziagdT1C+ZpZcrJjappBCDSw=
-github.com/go-git/go-git/v5 v5.0.0 h1:k5RWPm4iJwYtfWoxIJy4wJX9ON7ihPeZZYC1fLYDnpg=
-github.com/go-git/go-git/v5 v5.0.0/go.mod h1:oYD8y9kWsGINPFJoLdaScGCN6dlKg23blmClfZwtUVA=
-github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
-github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
-github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
-github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
-github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY=
-github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
-github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
-github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
-github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
-github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
-github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
-github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
-github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
-github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
-github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
-github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
-github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
-github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
-github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
-github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
-github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
-github.com/golang/protobuf v1.4.0 h1:oOuy+ugB+P/kBdUnG5QaMXSIyJ1q38wWSojYCb3z5VQ=
-github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
-github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
-github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
-github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
-github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
-github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
-github.com/google/go-cmp v0.4.0 h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4=
-github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4rEjNlfyDHW9dolSY=
-github.com/google/go-github/v28 v28.1.1 h1:kORf5ekX5qwXO2mGzXXOjMe/g6ap8ahVe0sBEulhSxo=
-github.com/google/go-github/v28 v28.1.1/go.mod h1:bsqJWQX05omyWVmc00nEUql9mhQyv38lDZ8kPZcQVoM=
-github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
-github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
-github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
-github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
-github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
-github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
-github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
-github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
-github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
-github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
-github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
-github.com/googleapis/gax-go/v2 v2.0.5 h1:sjZBwGj9Jlw33ImPtvFviGYvseOtDM7hkSKB7+Tv3SM=
-github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
-github.com/gorilla/handlers v1.4.2 h1:0QniY0USkHQ1RGCLfKxeNHK9bkDHGRYGNDFBCS+YARg=
-github.com/gorilla/handlers v1.4.2/go.mod h1:Qkdc/uu4tH4g6mTK6auzZ766c4CA0Ng8+o/OAirnOIQ=
-github.com/gorilla/mux v1.7.4 h1:VuZ8uybHlWmqV03+zRzdwKL4tUnIp1MAQtp1mIFE1bc=
-github.com/gorilla/mux v1.7.4/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
-github.com/h2non/filetype v1.0.12 h1:yHCsIe0y2cvbDARtJhGBTD2ecvqMSTvlIcph9En/Zao=
-github.com/h2non/filetype v1.0.12/go.mod h1:319b3zT68BvV+WRj7cwy856M2ehB3HqNOt6sy1HndBY=
-github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
-github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
-github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
-github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
-github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
-github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
-github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
-github.com/jstemmer/go-junit-report v0.9.1 h1:6QPYqodiu3GuPL+7mfx+NwDdp2eTkp9IfEUpgAwUN0o=
-github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
-github.com/kevinburke/ssh_config v0.0.0-20190725054713-01f96b0aa0cd h1:Coekwdh0v2wtGp9Gmz1Ze3eVRAWJMLokvN3QjdzCHLY=
-github.com/kevinburke/ssh_config v0.0.0-20190725054713-01f96b0aa0cd/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF4nAY/ojJ6r6mM=
-github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
-github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
-github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
-github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
-github.com/kr/pty v1.1.8/go.mod h1:O1sed60cT9XZ5uDucP5qwvh+TE3NnUj51EiZO/lmSfw=
-github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
-github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
-github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
-github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
-github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
-github.com/opencontainers/go-digest v1.0.0-rc1 h1:WzifXhOVOEOuFYOJAW6aQqW0TooG2iki3E3Ii+WN7gQ=
-github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
-github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
-github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
-github.com/pelletier/go-buffruneio v0.2.0/go.mod h1:JkE26KsDizTr40EUHkXVtNPvgGtbSNq5BcowyYOWdKo=
-github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
-github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
-github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
-github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
-github.com/satori/go.uuid v1.2.0 h1:0uYX9dsZ2yD7q2RtLRtPSdGDWzjeM3TbMJP9utgA0ww=
-github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
-github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
-github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
-github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
-github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
-github.com/src-d/gcfg v1.4.0 h1:xXbNR5AlLSA315x2UO+fTSSAXCDf+Ar38/6oyGbDKQ4=
-github.com/src-d/gcfg v1.4.0/go.mod h1:p/UMsR43ujA89BJY9duynAwIpvqEujIH/jFlfL7jWoI=
-github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
-github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
-github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
-github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
-github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
-github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
-github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
-github.com/xanzy/ssh-agent v0.2.1 h1:TCbipTQL2JiiCprBWx9frJ2eJlCYT00NmctrHxVAr70=
-github.com/xanzy/ssh-agent v0.2.1/go.mod h1:mLlQY/MoOhWBj+gOGMQkOeiEvkx+8pJSI+0Bx9h2kr4=
-github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
-github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
-go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
-go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
-go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
-go.opencensus.io v0.22.3 h1:8sGtKOrtQqkN1bp2AtX+misvLIlOmsEsNd+9NIcPEm8=
-go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
-golang.org/x/crypto v0.0.0-20190219172222-a4c6cb3142f2/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
-golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
-golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
-golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
-golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
-golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
-golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
-golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79 h1:IaQbIIB2X/Mp/DKctl6ROxz1KyMlKp4uyvL6+kQ7C88=
-golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
-golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
-golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
-golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
-golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
-golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
-golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
-golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
-golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
-golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
-golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
-golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
-golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
-golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
-golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
-golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
-golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
-golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
-golang.org/x/lint v0.0.0-20200302205851-738671d3881b h1:Wh+f8QHJXR411sJR8/vRBTZ7YapZaRvUcLFFJhusH0k=
-golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
-golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
-golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
-golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
-golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
-golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
-golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
-golang.org/x/mod v0.2.0 h1:KU7oHjnv3XNWfa5COkzUifxZmxp1TyI7ImMXqFxLwvQ=
-golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
-golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
-golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5 h1:WQ8q63x+f/zpC8Ac1s9wLElVoHhm32p6tudrU72n1QA=
-golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
-golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
-golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw=
-golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a h1:WXEvlFVvvGxCJLG6REjsT03iWnKLEWinaScsxF2Vm2o=
-golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190221075227-b4e8571b14e0/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200409092240-59c9f1ba88fa/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200501052902-10377860bb8e h1:hq86ru83GdWTlfQFZGO4nZJTU4Bs2wfHl8oFHRaXsfc=
-golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
-golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
-golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
-golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
-golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
-golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
-golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
-golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
-golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
-golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
-golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
-golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
-golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
-golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
-golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
-golang.org/x/tools v0.0.0-20190729092621-ff9f1409240a/go.mod h1:jcCCGcm9btYwXyDqrUWc6MKQKKGJCWEQ3AfLSRIbEuI=
-golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
-golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
-golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
-golang.org/x/tools v0.0.0-20200409170454-77362c5149f0/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
-golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d h1:lzLdP95xJmMpwQ6LUHwrc5V7js93hTiY7gkznu0BgmY=
-golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
-golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
-golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
-google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
-google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
-google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
-google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
-google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
-google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
-google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.21.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.23.0 h1:YlvGEOq2NA2my8cZ/9V8BcEO9okD48FlJcdqN0xJL3s=
-google.golang.org/api v0.23.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
-google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
-google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
-google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
-google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
-google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
-google.golang.org/appengine v1.6.6 h1:lMO5rYAqUxkmaj76jAkRUvt5JZgFymx/+Q5Mzfivuhc=
-google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
-google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
-google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
-google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
-google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
-google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
-google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
-google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
-google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
-google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
-google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
-google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200409111301-baae70f3302d/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/genproto v0.0.0-20200507105951-43844f6eee31 h1:Bz1qTn2YRWV+9OKJtxHJiQKCiXIdf+kwuKXdt9cBxyU=
-google.golang.org/genproto v0.0.0-20200507105951-43844f6eee31/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
-google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
-google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
-google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
-google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
-google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
-google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
-google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
-google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
-google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
-google.golang.org/grpc v1.28.1/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
-google.golang.org/grpc v1.29.1 h1:EC2SB8S04d2r73uptxphDSUG+kTKVgjRPF+N3xpxRB4=
-google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
-google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
-google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
-google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
-google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
-google.golang.org/protobuf v1.21.0 h1:qdOKuR/EIArgaWNjetjgTzgVTAZ+S/WXVrq9HW9zimw=
-google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
-gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
-gopkg.in/src-d/go-billy.v4 v4.3.2 h1:0SQA1pRztfTFx2miS8sA97XvooFeNOmvUenF4o0EcVg=
-gopkg.in/src-d/go-billy.v4 v4.3.2/go.mod h1:nDjArDMp+XMs1aFAESLRjfGSgfvoYN0hDfzEk0GjC98=
-gopkg.in/src-d/go-git-fixtures.v3 v3.5.0/go.mod h1:dLBcvytrw/TYZsNTWCnkNF2DSIlzWYqTe3rJR56Ac7g=
-gopkg.in/src-d/go-git.v4 v4.13.1 h1:SRtFyV8Kxc0UP7aCHcijOMQGPxHSmMOPrzulQWolkYE=
-gopkg.in/src-d/go-git.v4 v4.13.1/go.mod h1:nx5NYcxdKxq5fpltdHnPa2Exj4Sx0EclMWZQbYDu2z8=
-gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME=
-gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
-gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
-gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v3 v3.0.0-20200506231410-2ff61e1afc86 h1:OfFoIUYv/me30yv7XlMy4F9RJw8DEm8WQ6QG1Ph4bH0=
-gopkg.in/yaml.v3 v3.0.0-20200506231410-2ff61e1afc86/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
-honnef.co/go/tools v0.0.1-2020.1.3 h1:sXmLre5bzIR6ypkjXCDI3jHPssRhc8KD/Ome589sc3U=
-honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
-rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
-rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
-rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=

backend/go-app/main_test.go

@@ -0,0 +1,340 @@
+package main
+
+import (
+	"github.com/shuffle/shuffle-shared"
+
+	"bytes"
+	"context"
+	"log"
+	"net/http"
+	"net/http/httptest"
+	"reflect"
+	"runtime"
+	"testing"
+	"time"
+
+	"cloud.google.com/go/datastore"
+	"cloud.google.com/go/storage"
+	"google.golang.org/api/option"
+
+	"google.golang.org/grpc"
+)
+
+type endpoint struct {
+	handler http.HandlerFunc
+	path    string
+	method  string
+}
+
+func init() {
+	ctx := context.Background()
+	dbclient, err := datastore.NewClient(ctx, gceProject, option.WithGRPCDialOption(grpc.WithNoProxy()))
+	if err != nil {
+		log.Fatalf("[DEBUG] Database client error during init: %s", err)
+	}
+
+	_, err = shuffle.RunInit(*dbclient, storage.Client{}, gceProject, "onprem", true, "elasticsearch")
+	log.Printf("INIT")
+}
+
+// TestTestAuthenticationRequired tests that the handlers in the `handlers`
+// variable returns 401 Unauthorized when called without credentials.
+func TestAuthenticationRequired(t *testing.T) {
+	handlers := []endpoint{
+		{handler: shuffle.HandleNewOutlookRegister, path: "/functions/outlook/register", method: "GET"},
+		{handler: shuffle.HandleGetOutlookFolders, path: "/functions/outlook/getFolders", method: "GET"},
+		{handler: shuffle.HandleApiGeneration, path: "/api/v1/users/generateapikey", method: "GET"},
+		{handler: handleLogin, path: "/api/v1/users/login", method: "POST"}, // prob not this one
+		// handleRegister generates nil pointer exception. Not necessary for this anyway.
+		//{handler: handleRegister, path: "/api/v1/users/register", method: "POST"},
+		{handler: shuffle.HandleGetUsers, path: "/api/v1/users/getusers", method: "GET"},
+		{handler: handleInfo, path: "/api/v1/users/getinfo", method: "GET"},
+		{handler: shuffle.HandleSettings, path: "/api/v1/users/getsettings", method: "GET"},
+		{handler: shuffle.HandleUpdateUser, path: "/api/v1/users/updateuser", method: "PUT"},
+		{handler: shuffle.DeleteUser, path: "/api/v1/users/123", method: "DELETE"},
+		{handler: shuffle.HandlePasswordChange, path: "/api/v1/users/passwordchange", method: "POST"},
+		{handler: shuffle.HandleGetUsers, path: "/api/v1/users", method: "GET"},
+		{handler: shuffle.HandleGetEnvironments, path: "/api/v1/getenvironments", method: "GET"},
+		{handler: shuffle.HandleSetEnvironments, path: "/api/v1/setenvironments", method: "PUT"},
+
+		// handleWorkflowQueue generates nil pointer exception
+		//{handler: handleWorkflowQueue, path: "/api/v1/streams", method: "POST"},
+		// handleGetStreamResults generates nil pointer exception
+		//{handler: handleGetStreamResults, path: "/api/v1/streams/results", method: "POST"},
+
+		{handler: handleAppHotloadRequest, path: "/api/v1/apps/run_hotload", method: "GET"},
+		{handler: LoadSpecificApps, path: "/api/v1/apps/get_existing", method: "POST"},
+		{handler: shuffle.UpdateWorkflowAppConfig, path: "/api/v1/apps/123", method: "PATCH"},
+		{handler: validateAppInput, path: "/api/v1/apps/validate", method: "POST"},
+		{handler: shuffle.DeleteWorkflowApp, path: "/api/v1/apps/123", method: "DELETE"},
+		{handler: shuffle.GetWorkflowAppConfig, path: "/api/v1/apps/123/config", method: "GET"},
+		{handler: getWorkflowApps, path: "/api/v1/apps", method: "GET"},
+		{handler: setNewWorkflowApp, path: "/api/v1/apps", method: "PUT"},
+		//{handler: shuffle.GetSpecificApps, path: "/api/v1/apps/search", method: "POST"},
+
+		{handler: shuffle.GetAppAuthentication, path: "/api/v1/apps/authentication", method: "GET"},
+		{handler: shuffle.AddAppAuthentication, path: "/api/v1/apps/authentication", method: "PUT"},
+		{handler: shuffle.DeleteAppAuthentication, path: "/api/v1/apps/authentication/123", method: "DELETE"},
+
+		{handler: validateAppInput, path: "/api/v1/workflows/apps/validate", method: "POST"},
+		{handler: getWorkflowApps, path: "/api/v1/workflows/apps", method: "GET"},
+		{handler: setNewWorkflowApp, path: "/api/v1/workflows/apps", method: "PUT"},
+
+		{handler: shuffle.GetWorkflows, path: "/api/v1/workflows", method: "GET"},
+		{handler: shuffle.SetNewWorkflow, path: "/api/v1/workflows", method: "POST"},
+		{handler: handleGetWorkflowqueue, path: "/api/v1/workflows/queue", method: "GET"},
+		{handler: handleGetWorkflowqueueConfirm, path: "/api/v1/workflows/queue/confirm", method: "POST"},
+		{handler: shuffle.HandleGetSchedules, path: "/api/v1/workflows/schedules", method: "GET"},
+		{handler: loadSpecificWorkflows, path: "/api/v1/workflows/download_remote", method: "POST"},
+		{handler: executeWorkflow, path: "/api/v1/workflows/123/execute", method: "GET"},
+		{handler: scheduleWorkflow, path: "/api/v1/workflows/123/schedule", method: "POST"},
+		{handler: stopSchedule, path: "/api/v1/workflows/123/schedule/abc", method: "DELETE"},
+		// createOutlookSub generates nil pointer exception
+		{handler: shuffle.HandleCreateOutlookSub, path: "/api/v1/workflows/123/outlook", method: "POST"},
+		// handleDeleteOutlookSub generates nil pointer exception
+		{handler: shuffle.HandleDeleteOutlookSub, path: "/api/v1/workflows/123/outlook/abc", method: "DELETE"},
+		{handler: shuffle.GetWorkflowExecutions, path: "/api/v1/workflows/123/executions", method: "GET"},
+		{handler: shuffle.AbortExecution, path: "/api/v1/workflows/123/executions/abc/abort", method: "GET"},
+		{handler: shuffle.GetSpecificWorkflow, path: "/api/v1/workflows/123", method: "GET"},
+		{handler: shuffle.SaveWorkflow, path: "/api/v1/workflows/123", method: "PUT"},
+		{handler: deleteWorkflow, path: "/api/v1/workflows/123", method: "DELETE"},
+
+		{handler: shuffle.HandleNewHook, path: "/api/v1/hooks/new", method: "POST"},
+		{handler: handleWebhookCallback, path: "/api/v1/hooks/123", method: "POST"},
+		{handler: shuffle.HandleDeleteHook, path: "/api/v1/hooks/123/delete", method: "DELETE"},
+
+		{handler: shuffle.HandleGetSpecificTrigger, path: "/api/v1/triggers/123", method: "GET"},
+		//{handler: shuffle.HandleGetSpecificStats, path: "/api/v1/stats/123", method: "GET"},
+
+		{handler: verifySwagger, path: "/api/v1/verify_swagger", method: "POST"},
+		{handler: verifySwagger, path: "/api/v1/verify_openapi", method: "POST"},
+		{handler: echoOpenapiData, path: "/api/v1/get_openapi_uri", method: "POST"},
+		{handler: echoOpenapiData, path: "/api/v1/validate_openapi", method: "POST"},
+		{handler: shuffle.ValidateSwagger, path: "/api/v1/validate_openapi", method: "POST"},
+		{handler: getOpenapi, path: "/api/v1/get_openapi", method: "GET"},
+
+		//{handler: shuffle.CleanupExecutions, path: "/api/v1/execution_cleanup", method: "GET"},
+
+		{handler: handleCloudSetup, path: "/api/v1/cloud/setup", method: "POST"},
+		{handler: shuffle.HandleGetOrgs, path: "/api/v1/orgs", method: "POST"},
+		{handler: shuffle.HandleGetFileContent, path: "/api/v1/files/{fileId}/content", method: "POST"},
+	}
+
+	var err error
+	ctx := context.Background()
+
+	// Most handlers requires database access in order to not crash or cause
+	// nil pointer issues.
+	// To start a local database instance, run:
+	//   docker-compose up database
+	// To let the tests know about the database, run:
+	//   DATASTORE_EMULATOR_HOST=0.0.0.0:8000 go test
+	dbclient, err = datastore.NewClient(ctx, gceProject, option.WithGRPCDialOption(grpc.WithNoProxy()))
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	dummyBody := bytes.NewBufferString("dummy")
+
+	for _, e := range handlers {
+		log.Printf("Endpoint: %#v", e.path)
+		req, err := http.NewRequest(e.method, e.path, dummyBody)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		rr := httptest.NewRecorder()
+		handler := http.HandlerFunc(e.handler)
+
+		timeoutHandler := http.TimeoutHandler(handler, 2*time.Second, `Request Timeout.`)
+		timeoutHandler.ServeHTTP(rr, req)
+
+		funcName := getFunctionNameFromFunction(e.handler)
+		if status := rr.Code; status != http.StatusUnauthorized {
+			t.Errorf("%s handler returned wrong status code: got %v want %v",
+				funcName, status, http.StatusUnauthorized)
+		}
+	}
+}
+
+func TestAuthenticationNotRequired(t *testing.T) {
+	// All of these return 200 OK when user not logged in
+	handlers := []endpoint{
+		{handler: checkAdminLogin, path: "/api/v1/users/checkusers", method: "GET"},
+		{handler: shuffle.HandleLogout, path: "/api/v1/users/logout", method: "POST"},
+		{handler: shuffle.GetDocList, path: "/api/v1/docs", method: "GET"},
+		{handler: shuffle.GetDocs, path: "/api/v1/docs/123", method: "GET"},
+		{handler: healthCheckHandler, path: "/api/v1/_ah/health"},
+	}
+
+	for _, e := range handlers {
+		log.Printf("Endpoint: %#v", e.path)
+		req, err := http.NewRequest(e.method, e.path, nil)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		rr := httptest.NewRecorder()
+		handler := http.HandlerFunc(e.handler)
+
+		timeoutHandler := http.TimeoutHandler(handler, 2*time.Second, `Request Timeout.`)
+		timeoutHandler.ServeHTTP(rr, req)
+
+		funcName := getFunctionNameFromFunction(e.handler)
+		if status := rr.Code; status != http.StatusOK {
+			t.Errorf("%s handler returned wrong status code: got %v want %v",
+				funcName, status, http.StatusOK)
+		}
+	}
+}
+
+// TestCors tests that all endpoints returns the same CORS headers when hit
+// with an OPTIONS type request.
+// It feels very fragile to test headers like this, especially for the
+// "Access-Control-Allow-Origin", but this test should be helpful while
+// refactoring the CORS logic into a middleware, and that's the reason this
+// test exists right now. It might change after the refactor because our
+// requirements might change after the refactor.
+func TestCors(t *testing.T) {
+	handlers := []endpoint{
+		{handler: shuffle.HandleNewOutlookRegister, path: "/functions/outlook/register", method: "GET"},
+		{handler: shuffle.HandleGetOutlookFolders, path: "/functions/outlook/getFolders", method: "GET"},
+		{handler: shuffle.HandleApiGeneration, path: "/api/v1/users/generateapikey", method: "GET"},
+		{handler: handleLogin, path: "/api/v1/users/login", method: "POST"}, // prob not this one
+		// handleRegister generates nil pointer exception
+		{handler: handleRegister, path: "/api/v1/users/register", method: "POST"},
+		{handler: shuffle.HandleGetUsers, path: "/api/v1/users/getusers", method: "GET"},
+		{handler: handleInfo, path: "/api/v1/users/getinfo", method: "GET"},
+		{handler: shuffle.HandleSettings, path: "/api/v1/users/getsettings", method: "GET"},
+		{handler: shuffle.HandleUpdateUser, path: "/api/v1/users/updateuser", method: "PUT"},
+		{handler: shuffle.DeleteUser, path: "/api/v1/users/123", method: "DELETE"},
+		// handlePasswordChange generates nil pointer exception
+		{handler: shuffle.HandlePasswordChange, path: "/api/v1/users/passwordchange", method: "POST"},
+		{handler: shuffle.HandleGetUsers, path: "/api/v1/users", method: "GET"},
+		{handler: shuffle.HandleGetEnvironments, path: "/api/v1/getenvironments", method: "GET"},
+		{handler: shuffle.HandleSetEnvironments, path: "/api/v1/setenvironments", method: "PUT"},
+
+		// handleWorkflowQueue generates nil pointer exception
+		{handler: handleWorkflowQueue, path: "/api/v1/streams", method: "POST"},
+		// handleGetStreamResults generates nil pointer exception
+		{handler: handleGetStreamResults, path: "/api/v1/streams/results", method: "POST"},
+
+		{handler: handleAppHotloadRequest, path: "/api/v1/apps/run_hotload", method: "GET"},
+		{handler: LoadSpecificApps, path: "/api/v1/apps/get_existing", method: "POST"},
+		{handler: shuffle.UpdateWorkflowAppConfig, path: "/api/v1/apps/123", method: "PATCH"},
+		{handler: validateAppInput, path: "/api/v1/apps/validate", method: "POST"},
+		{handler: shuffle.DeleteWorkflowApp, path: "/api/v1/apps/123", method: "DELETE"},
+		{handler: shuffle.GetWorkflowAppConfig, path: "/api/v1/apps/123/config", method: "GET"},
+		{handler: getWorkflowApps, path: "/api/v1/apps", method: "GET"},
+		{handler: setNewWorkflowApp, path: "/api/v1/apps", method: "PUT"},
+		//{handler: shuffle.GetSpecificApps, path: "/api/v1/apps/search", method: "POST"},
+
+		{handler: shuffle.GetAppAuthentication, path: "/api/v1/apps/authentication", method: "GET"},
+		{handler: shuffle.AddAppAuthentication, path: "/api/v1/apps/authentication", method: "PUT"},
+		{handler: shuffle.DeleteAppAuthentication, path: "/api/v1/apps/authentication/123", method: "DELETE"},
+
+		{handler: validateAppInput, path: "/api/v1/workflows/apps/validate", method: "POST"},
+		{handler: getWorkflowApps, path: "/api/v1/workflows/apps", method: "GET"},
+		{handler: setNewWorkflowApp, path: "/api/v1/workflows/apps", method: "PUT"},
+
+		{handler: shuffle.GetWorkflows, path: "/api/v1/workflows", method: "GET"},
+		{handler: shuffle.SetNewWorkflow, path: "/api/v1/workflows", method: "POST"},
+		{handler: handleGetWorkflowqueue, path: "/api/v1/workflows/queue", method: "GET"},
+		{handler: handleGetWorkflowqueueConfirm, path: "/api/v1/workflows/queue/confirm", method: "POST"},
+		{handler: shuffle.HandleGetSchedules, path: "/api/v1/workflows/schedules", method: "GET"},
+		{handler: loadSpecificWorkflows, path: "/api/v1/workflows/download_remote", method: "POST"},
+		{handler: executeWorkflow, path: "/api/v1/workflows/123/execute", method: "GET"},
+		{handler: scheduleWorkflow, path: "/api/v1/workflows/123/schedule", method: "POST"},
+		{handler: stopSchedule, path: "/api/v1/workflows/123/schedule/abc", method: "DELETE"},
+		// createOutlookSub generates nil pointer exception
+		{handler: shuffle.HandleCreateOutlookSub, path: "/api/v1/workflows/123/outlook", method: "POST"},
+		// handleDeleteOutlookSub generates nil pointer exception
+		{handler: shuffle.HandleDeleteOutlookSub, path: "/api/v1/workflows/123/outlook/abc", method: "DELETE"},
+		{handler: shuffle.GetWorkflowExecutions, path: "/api/v1/workflows/123/executions", method: "GET"},
+		{handler: shuffle.AbortExecution, path: "/api/v1/workflows/123/executions/abc/abort", method: "GET"},
+		{handler: shuffle.GetSpecificWorkflow, path: "/api/v1/workflows/123", method: "GET"},
+		{handler: shuffle.SaveWorkflow, path: "/api/v1/workflows/123", method: "PUT"},
+		{handler: deleteWorkflow, path: "/api/v1/workflows/123", method: "DELETE"},
+
+		{handler: shuffle.HandleNewHook, path: "/api/v1/hooks/new", method: "POST"},
+		{handler: handleWebhookCallback, path: "/api/v1/hooks/123", method: "POST"},
+		{handler: shuffle.HandleDeleteHook, path: "/api/v1/hooks/123/delete", method: "DELETE"},
+
+		{handler: shuffle.HandleGetSpecificTrigger, path: "/api/v1/triggers/123", method: "GET"},
+		//{handler: shuffle.HandleGetSpecificStats, path: "/api/v1/stats/123", method: "GET"},
+
+		{handler: verifySwagger, path: "/api/v1/verify_swagger", method: "POST"},
+		{handler: verifySwagger, path: "/api/v1/verify_openapi", method: "POST"},
+		{handler: echoOpenapiData, path: "/api/v1/get_openapi_uri", method: "POST"},
+		{handler: echoOpenapiData, path: "/api/v1/validate_openapi", method: "POST"},
+		{handler: shuffle.ValidateSwagger, path: "/api/v1/validate_openapi", method: "POST"},
+		{handler: getOpenapi, path: "/api/v1/get_openapi", method: "GET"},
+
+		//{handler: shuffle.CleanupExecutions, path: "/api/v1/execution_cleanup", method: "GET"},
+
+		{handler: handleCloudSetup, path: "/api/v1/cloud/setup", method: "POST"},
+		{handler: shuffle.HandleGetOrgs, path: "/api/v1/orgs", method: "POST"},
+		{handler: shuffle.HandleGetFileContent, path: "/api/v1/files/{fileId}/content", method: "POST"},
+	}
+
+	//r := initHandlers(context.TODO())
+	initHandlers()
+
+outerLoop:
+	for _, e := range handlers {
+		log.Printf("Endpoint: %#v", e.path)
+		req, err := http.NewRequest("OPTIONS", e.path, nil)
+		req.Header.Add("Origin", "http://localhost:3000")
+		req.Header.Add("Access-Control-Request-Method", "POST")
+		req.Header.Add("Access-Control-Request-Headers", "Content-Type, Accept, X-Requested-With, remember-me")
+
+		// OPTIONS /resource/foo
+		// Access-Control-Request-Method: DELETE
+		// Access-Control-Request-Headers: origin, x-requested-with
+		// Origin: https://foo.bar.org
+
+		if err != nil {
+			t.Errorf("Failure in OPTIONS setup: %s", err)
+			continue
+		}
+
+		rr := httptest.NewRecorder()
+
+		//timeoutHandler := http.TimeoutHandler(r, 2*time.Second, `Request Timeout`)
+		//timeoutHandler.ServeHTTP(rr, req)
+
+		funcName := getFunctionNameFromFunction(e.handler)
+		if status := rr.Code; status != http.StatusOK {
+			t.Errorf("%s handler returned wrong status code: got %v want %v",
+				funcName, status, http.StatusOK)
+			continue
+		}
+
+		want := map[string]string{
+			"Vary":                             "Origin",
+			"Access-Control-Allow-Headers":     "Content-Type, Accept, X-Requested-With, Remember-Me",
+			"Access-Control-Allow-Methods":     "POST",
+			"Access-Control-Allow-Credentials": "true",
+			"Access-Control-Allow-Origin":      "http://localhost:3000",
+		}
+
+		// Remember to use canonical header name if accessing the headers array
+		// directly:
+		//   v := r.Header[textproto.CanonicalMIMEHeaderKey("foo")]
+		// When using Header().Get(h), h will automatically be converted to canonical format.
+
+		for key, value := range want {
+			got := rr.Header().Get(key)
+			if got != value {
+				t.Errorf("%s handler returned wrong value for '%s' header: got '%v' want '%v'",
+					funcName, key, got, value)
+				continue outerLoop
+			}
+		}
+
+	}
+}
+
+func getFunctionNameFromFunction(f interface{}) string {
+	return runtime.FuncForPC(reflect.ValueOf(f).Pointer()).Name()
+}

backend/go-app/run

@@ -0,0 +1 @@
+go run main.go walkoff.go docker.go

backend/tests/cache.sh

@@ -0,0 +1,15 @@
+curl -XPOST http://192.168.3.8:5001/api/v1/orgs/6a6a99f5-6630-4f91-88ff-571c9f030ea0/set_cache -H "Authorization: Bearer e663cf93-7f10-4560-bef0-303f14aad982" -d '{
+	"workflow_id": "61825389-a125-43a5-9119-97c401e9934b",
+	"execution_id": "2c8ca1b7-6658-4742-86a1-105c9467702d",
+	"org_id": "6a6a99f5-6630-4f91-88ff-571c9f030ea0",
+	"key": "test",
+	"value": "THIS IS SOME DATA HELLO"
+}'
+
+curl -XPOST http://192.168.3.8:5001/api/v1/orgs/6a6a99f5-6630-4f91-88ff-571c9f030ea0/get_cache -H "Authorization: Bearer e663cf93-7f10-4560-bef0-303f14aad982" -d '{
+	"workflow_id": "61825389-a125-43a5-9119-97c401e9934b",
+	"execution_id": "2c8ca1b7-6658-4742-86a1-105c9467702d",
+	"org_id": "6a6a99f5-6630-4f91-88ff-571c9f030ea0",
+	"key": "test"
+}'
+

backend/tests/dockerpull.sh

@@ -0,0 +1,5 @@
+#!/bin/sh
+#curl -XPOST http://localhost:5001/api/v1/get_docker_image -d '{"name":"frikky/shuffle:testing_1.0.0"}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" --output tarball.tgz
+#curl -XPOST http://localhost:5001/api/v1/get_docker_image -d '{"name":"frikky/shuffle:testing_1.0.0"}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -O -J
+#curl -XPOST http://localhost:5001/api/v1/get_docker_image -d '{"name":"frikky/shuffle:testing_1.0.0"}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" --output tarball.tgz
+curl -XPOST http://localhost:5001/api/v1/get_docker_image -d '{"name":"frikky/shuffle:shuffle-tools_1.0.0"}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" --output tarball.tgz

backend/tests/files.sh

@@ -0,0 +1,22 @@
+#curl http://192.168.3.6:5001/api/v1/files/create -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -d '{"filename": "file.txt", "org_id": "b199646b-16d2-456d-9fd6-b9972e929466", "workflow_id": "global"}'
+#
+#curl http://localhost:5001/api/v1/files/create -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -d '{"filename": "file.txt", "org_id": "b199646b-16d2-456d-9fd6-b9972e929466", "workflow_id": "global"}'
+#
+#echo
+curl http://localhost:5001/api/v1/apps/upload -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -F 'shuffle_file=@files.sh'
+#
+#curl http://localhost:5001/api/v1/files/1915981b-b897-4db1-8a2e-44bc34cead3b/content -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" 
+#curl http://localhost:5001/api/v1/files/e19cffe4-e2da-47e9-809e-904f5cb03687 -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" 
+#curl -XDELETE http://localhost:5001/api/v1/files/e19cffe4-e2da-47e9-809e-904f5cb03687 -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" 
+
+	#r.HandleFunc("/api/v1/files/{fileId}/content", handleGetFileContent).Methods("GET", "OPTIONS")
+	#r.HandleFunc("/api/v1/files/create", handleCreateFile).Methods("POST", "OPTIONS")
+	#r.HandleFunc("/api/v1/files/{fileId}/upload", handleUploadFile).Methods("POST", "OPTIONS")
+	#r.HandleFunc("/api/v1/files/{fileId}", handleGetFileMeta).Methods("GET", "OPTIONS")
+	#r.HandleFunc("/api/v1/files/{fileId}", handleDeleteFile).Methods("DELETE", "OPTIONS")
+
+
+#curl http://localhost:5001/api/v1/files/create -H "Authorization: Bearer c5b4c827-65ec-47f4-9e8a-234cdba38959" -d '{"filename": "rule2.yar", "org_id": "b4e88fe9-352b-47b4-b280-960181670acf", "workflow_id": "global", "namespace": "yara"}'
+#curl http://localhost:5001/api/v1/files/5cb941ad-fa1c-4444-a685-92024b1fa31c/upload -H "Authorization: Bearer c5b4c827-65ec-47f4-9e8a-234cdba38959" -F 'shuffle_file=@upload.sh'
+
+#curl http://localhost:5001/api/v1/files/namespaces/yara -H "Authorization: Bearer c5b4c827-65ec-47f4-9e8a-234cdba38959" --output rules.zip

backend/tests/hotload.sh

@@ -0,0 +1 @@
+curl http://localhost:5001/api/v1/apps/run_hotload -H "Authorization: Bearer e08c6f22-9a55-4557-b008-04388cc51fb0"

backend/tests/list_files.sh

@@ -0,0 +1 @@
+curl -XGET http://192.168.3.6:5001/api/v1/files -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"

backend/tests/migrate_db.sh

@@ -0,0 +1,2 @@
+curl -XPOST -v localhost:5001/api/v1/migrate_database -H 'Authorization: Bearer 0184d7be-33c1-4391-bf9c-dfb8508a4ea2'
+

backend/tests/schedules.sh

@@ -1,2 +1,2 @@
 # Fails cus of unmarshal
-curl -XPOST http://localhost:5001/api/v1/workflows/1d9d8ce2-566e-4c3f-8a37-5d6c7d2000b5/schedule -d '{"name": "hey", "frequency": "*/1 * * * *", "execution_argument": "{\"test\": \"hey\"}"}' -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IjYwZjQwNjBlNThkNzVmZDNmNzBiZWZmODhjNzk0YTc3NTMyN2FhMzEiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJodHRwczovL3NodWZmbGVyLmlvL2FwaS92MS93b3JrZmxvd3MvMWQ5ZDhjZTItNTY2ZS00YzNmLThhMzctNWQ2YzdkMjAwMGI1L2V4ZWN1dGUiLCJhenAiOiIxMDMwNzY3ODIwNjE0MjQ2MTg0MjIiLCJlbWFpbCI6InNjaGVkdWxlckBzaHVmZmxlLTI0MTUxNy5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJleHAiOjE1NjU1Mjc1NTEsImlhdCI6MTU2NTUyMzk1MSwiaXNzIjoiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tIiwic3ViIjoiMTAzMDc2NzgyMDYxNDI0NjE4NDIyIn0.r0EDq9fjhf_5CPTiltyfk_L3uYJp577Uy0yYPcCAl2nv50_z_oUtbWGBpQLL8gcj-NGd3g4E52Qur8k6hCMIQweLS6WAb1279vGffEoCNDfkWb3Oy-yJGP1kzwLvqFJqnHLkSWYXNWvSyWnEimW8Rryx_m1BXS5wcA8l4NIr83kS7fPZrTwjnwFSeGSThwk91DVARzapQb8r0GEgOUyHZ1aBXnV98mikzSUt-5xFKe9eMdD22YJAj0Ru-DxAxs5nOqghX4PMRysWjshjOMrlR1piPWxqAmewp8YKZDCQ5gXskpeAFBDoULT971Wsx_NCohnJsFqx1JfPS9ZYMTW2oQ"
+curl -XPOST http://localhost:5001/api/v1/workflows/1d9d8ce2-566e-4c3f-8a37-5d6c7d2000b5/schedule -d '{"name": "hey", "frequency": "*/1 * * * *", "execution_argument": "{\"test\": \"hey\"}"}' -H "Authorization: Bearer WUT"

backend/tests/stop_execution.sh

@@ -0,0 +1 @@
+curl "http://localhost:5001/api/v1/environments/Shuffle/stop" -H "Authorization: Bearer e663cf93-7f10-4560-bef0-303f14aad982"

backend/tests/test_wrappers.py

@@ -0,0 +1,210 @@
+import re
+import json
+
+def parse_nested_param(string, level):
+    """
+    Generate strings contained in nested (), indexing i = level
+    """
+    if len(re.findall("\(", string)) == len(re.findall("\)", string)):
+        LeftRightIndex = [x for x in zip(
+        [Left.start()+1 for Left in re.finditer('\(', string)], 
+        reversed([Right.start() for Right in re.finditer('\)', string)]))]
+
+    elif len(re.findall("\(", string)) > len(re.findall("\)", string)):
+        return parse_nested_param(string + ')', level)
+    elif len(re.findall("\(", string)) < len(re.findall("\)", string)):
+        return parse_nested_param('(' + string, level)
+
+    else:
+        return 'Failed to parse params'
+
+    try:
+        return [string[LeftRightIndex[level][0]:LeftRightIndex[level][1]]]
+    except IndexError:
+        return [string[LeftRightIndex[level+1][0]:LeftRightIndex[level+1][1]]]
+
+# Parses the deepest part
+def maxDepth(S): 
+    current_max = 0
+    max = 0
+    n = len(S) 
+  
+    # Traverse the input string 
+    for i in range(n): 
+        if S[i] == '(': 
+            current_max += 1
+  
+            if current_max > max: 
+                max = current_max 
+        elif S[i] == ')': 
+            if current_max > 0: 
+                current_max -= 1
+            else: 
+                return -1
+  
+    # finally check for unbalanced string 
+    if current_max != 0: 
+        return -1
+  
+    return max-1
+
+def parse_type(data, thistype): 
+    if data == None:
+        return "Empty"
+
+    if "int" in thistype:
+        try:
+            return int(data)
+        except ValueError:
+            print("ValueError while casting %s" % data)
+            return data
+    if "lower" in thistype:
+        return data.lower()
+    if "upper" in thistype:
+        return data.upper()
+    if "trim" in thistype:
+        return data.strip()
+    if "strip" in thistype:
+        return data.strip()
+    if "split" in thistype:
+        # Should be able to split anything
+        return data.split()
+    if "len" in thistype or "length" in thistype:
+        return len(data)
+    if "parse" in thistype:
+        splitvalues = []
+        default_error = """Error. Expected syntax: parse(["hello","test1"],0:1)""" 
+        if "," in data:
+            splitvalues = data.split(",")
+
+            for item in range(len(splitvalues)):
+                splitvalues[item] = splitvalues[item].strip()
+        else:
+            return default_error 
+
+        lastsplit = []
+        if ":" in splitvalues[-1]:
+            lastsplit = splitvalues[-1].split(":")
+        else:
+            try:
+                lastsplit = [int(splitvalues[-1])]
+            except ValueError:
+                return default_error
+
+        try:
+            parsedlist = ",".join(splitvalues[0:-1])
+            print(parsedlist)
+            print(lastsplit)
+
+            if len(lastsplit) > 1:
+                tmp = json.loads(parsedlist)[int(lastsplit[0]):int(lastsplit[1])]
+            else:
+                tmp = json.loads(parsedlist)[lastsplit[0]]
+
+            print(tmp)
+            return tmp
+        except IndexError as e:
+            return default_error
+
+# Parses the INNER value and recurses until everything is done
+def parse_wrapper(data):
+    try:
+        if "(" not in data or ")" not in data:
+            return data
+    except TypeError:
+        return data
+
+    print("Running %s" % data)
+
+    # Look for the INNER wrapper first, then move out
+    wrappers = ["int", "number", "lower", "upper", "trim", "strip", "split", "parse", "len", "length"]
+    found = False
+    for wrapper in wrappers:
+        if wrapper not in data.lower():
+            continue
+
+        found = True
+        break
+
+    if not found:
+        return data
+
+    # Do stuff here.
+    innervalue = parse_nested_param(data, maxDepth(data)-0)
+    outervalue = parse_nested_param(data, maxDepth(data)-1)
+    print("INNER: ", outervalue)
+    print("OUTER: ", outervalue)
+
+    if outervalue != innervalue:
+        #print("Outer: ", outervalue, " inner: ", innervalue)
+        for key in range(len(innervalue)):
+            # Replace OUTERVALUE[key] with INNERVALUE[key] in data.
+            print("Replace %s with %s in %s" % (outervalue[key], innervalue[key], data))
+            data = data.replace(outervalue[key], innervalue[key])
+    else:
+        for thistype in wrappers:
+            if thistype not in data.lower():
+                continue
+
+            parsed_value = parse_type(innervalue[0], thistype)
+            return parsed_value
+
+    print("DATA: %s\n" % data)
+    return parse_wrapper(data)
+
+def parse_wrapper_start(data):
+    newdata = []
+    newstring = ""
+    record = True
+    paranCnt = 0
+    for char in data:
+        if char == "(":
+            paranCnt += 1
+
+            if not record:
+                record = True 
+
+        if record:
+            newstring += char
+
+        if paranCnt == 0 and char == " ":
+            newdata.append(newstring)
+            newstring = ""
+            record = True
+
+        if char == ")":
+            paranCnt -= 1
+
+            if paranCnt == 0:
+                record = False
+
+    if len(newstring) > 0:
+        newdata.append(newstring)
+
+    parsedlist = []
+    non_string = False
+    for item in newdata:
+        ret = parse_wrapper(item)
+        if not isinstance(ret, str):
+            non_string = True
+
+        parsedlist.append(ret)
+
+    if len(parsedlist) > 0 and not non_string:
+        return " ".join(parsedlist)
+    elif len(parsedlist) == 1 and non_string:
+        return parsedlist[0]
+    else:
+        print("Casting back to string because multi: ", parsedlist)
+        newlist = []
+        for item in parsedlist:
+            try:
+                newlist.append(str(item))
+            except ValueError:
+                newlist.append("parsing_error")
+        return " ".join(newlist)
+
+data = "split(hello there)"
+data = """parse(["testing", "what", "is this"], 0:2)"""
+#data = "int(int(2))"
+print("RET: ", parse_wrapper_start(data))

backend/tests/upload.sh

@@ -0,0 +1 @@
+#

backend/tests/users.sh

@@ -0,0 +1,13 @@
+curl http://localhost:5001/api/v1/users/register -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" -d '{"username": "username1", "password": ""}'
+
+echo
+curl http://localhost:5001/api/v1/users -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" 
+
+echo UPDATE
+curl -XPUT http://localhost:5001/api/v1/users/updateuser -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" -d '{"user_id": "id", "role": "admin"}'
+
+echo 
+curl -XDELETE http://localhost:5001/api/v1/users/userid -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" 
+
+echo
+curl -XPOST http://localhost:5001/api/v1/users/generateapikey -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" -d '{"user_id": "390efa79-73a3-454b-8b1d-38f56eec14ad"}'

backend/tests/validate_app_values.sh

@@ -0,0 +1,14 @@
+# ExecutionOrg MUST be executing.
+curl -XPOST http://localhost:5001/api/v1/orgs/b199646b-16d2-456d-9fd6-b9972e929466/validate_app_values -d '{
+	"append": true,
+	"workflow_check": true,
+	"authorization": "1aae630c-ccaf-4cb5-87f9-8a9e0a9afd11",
+	"execution_ref": "c59ff288-4f02-4d02-b839-133d55c7fdf0",
+	"org_id": "b199646b-16d2-456d-9fd6-b9972e929466",
+	"values": [{
+		"app": "testing",
+		"action": "repeat_back_to_me",
+		"parameternames": ["call"],
+		"parametervalues": ["hey", "ho", "lets", "go"]
+	}]
+}'

docker-compose.yml

@@ -2,7 +2,7 @@ version: '3'
 services:
   frontend:
     #build: ./frontend
-    image: ghcr.io/frikky/shuffle-frontend:0.8.56
+    image: ghcr.io/frikky/shuffle-frontend:nightly
     container_name: shuffle-frontend
     hostname: shuffle-frontend
     ports:
@@ -16,8 +16,8 @@ services:
     depends_on:
       - backend
   backend:
-    #build: ./backend
+    build: ./backend
-    image: ghcr.io/frikky/shuffle-backend:0.8.56
+    image: ghcr.io/frikky/shuffle-backend:nightly
     container_name: shuffle-backend
     hostname: ${BACKEND_HOSTNAME}
     # Here for debugging:
@@ -25,27 +25,21 @@ services:
       - "${BACKEND_PORT}:5001"
     networks:
       - shuffle
-    volumes: 
+    volumes:
-      - /var/run/docker.sock:/var/run/docker.sock 
+      - /var/run/docker.sock:/var/run/docker.sock
-      - ${SHUFFLE_APP_HOTLOAD_LOCATION}:/shuffle-apps     
+      - ${SHUFFLE_APP_HOTLOAD_LOCATION}:/shuffle-apps:z
-      - ${SHUFFLE_FILE_LOCATION}:/shuffle-files
+      - ${SHUFFLE_FILE_LOCATION}:/shuffle-files:z
+      #- ${SHUFFLE_OPENSEARCH_CERTIFICATE_FILE}:/shuffle-files/es_certificate
+    env_file: .env
     environment:
-      - DATASTORE_EMULATOR_HOST=shuffle-database:8000
       - SHUFFLE_APP_HOTLOAD_FOLDER=/shuffle-apps
       - SHUFFLE_FILE_LOCATION=/shuffle-files
-      - ORG_ID=${ORG_ID}
-      - SHUFFLE_APP_DOWNLOAD_LOCATION=${SHUFFLE_APP_DOWNLOAD_LOCATION}
-      - SHUFFLE_DEFAULT_USERNAME=${SHUFFLE_DEFAULT_USERNAME}
-      - SHUFFLE_DEFAULT_PASSWORD=${SHUFFLE_DEFAULT_PASSWORD}
-      - SHUFFLE_DEFAULT_APIKEY=${SHUFFLE_DEFAULT_APIKEY}
-      - HTTP_PROXY=${SHUFFLE_HTTP_PROXY}
-      - HTTPS_PROXY=${SHUFFLE_HTTPS_PROXY}
     restart: unless-stopped
-    depends_on:
+    #depends_on:
-      - database
+      #- opensearch #Not necessary because dependancy is handled within the backend itself instead
+      #- database
   orborus:
-    #build: ./functions/onprem/orborus
+    image: ghcr.io/frikky/shuffle-orborus:nightly
-    image: ghcr.io/frikky/shuffle-orborus:0.8.5
     container_name: shuffle-orborus
     hostname: shuffle-orborus
     networks:
@@ -53,33 +47,76 @@ services:
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock
     environment:
-      - SHUFFLE_APP_SDK_VERSION=0.8.51
+      - SHUFFLE_WORKER_VERSION=nightly
-      - SHUFFLE_WORKER_VERSION=0.8.54
-      - ORG_ID=${ORG_ID}
       - ENVIRONMENT_NAME=${ENVIRONMENT_NAME}
-      - BASE_URL=http://${OUTER_HOSTNAME}:${BACKEND_PORT}
+      - BASE_URL=http://${OUTER_HOSTNAME}:5001
       - DOCKER_API_VERSION=1.40
-      - HTTP_PROXY=${SHUFFLE_HTTP_PROXY}
-      - HTTPS_PROXY=${SHUFFLE_HTTPS_PROXY}
-      - SHUFFLE_PASS_WORKER_PROXY=${SHUFFLE_PASS_WORKER_PROXY}
-      - SHUFFLE_ORBORUS_EXECUTION_TIMEOUT=600
       - SHUFFLE_BASE_IMAGE_NAME=${SHUFFLE_BASE_IMAGE_NAME}
       - SHUFFLE_BASE_IMAGE_REGISTRY=${SHUFFLE_BASE_IMAGE_REGISTRY}
       - SHUFFLE_BASE_IMAGE_TAG_SUFFIX=${SHUFFLE_BASE_IMAGE_TAG_SUFFIX}
-      - CLEANUP=${SHUFFLE_CONTAINER_AUTO_CLEANUP}
+      - HTTP_PROXY=${HTTP_PROXY}
+      - HTTPS_PROXY=${HTTPS_PROXY}
+      - SHUFFLE_PASS_WORKER_PROXY=${SHUFFLE_PASS_WORKER_PROXY}
+      - SHUFFLE_PASS_APP_PROXY=${SHUFFLE_PASS_APP_PROXY}
+      - SHUFFLE_SCALE_REPLICAS=1
+      - SHUFFLE_SWARM_CONFIG=runn
+      - SHUFFLE_SWARM_NETWORK_NAME=shuffle_swarm_executions
+      #- DOCKER_HOST=tcp://docker-socket-proxy:2375
     restart: unless-stopped
-  database:
+    security_opt:
-    #build: ./backend/database
+      - seccomp:unconfined
-    image: frikky/shuffle:database
+  opensearch:
-    container_name: shuffle-database
+    image: opensearchproject/opensearch:1.2.4
-    hostname: shuffle-database
+    hostname: shuffle-opensearch
+    container_name: shuffle-opensearch
+    environment:
+      - bootstrap.memory_lock=true
+      - "OPENSEARCH_JAVA_OPTS=-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
+      - cluster.routing.allocation.disk.threshold_enabled=false
+      - cluster.name=shuffle-cluster
+      - node.name=shuffle-opensearch
+      - discovery.seed_hosts=shuffle-opensearch
+      - cluster.initial_master_nodes=shuffle-opensearch
+      - node.store.allow_mmap=false
+    ulimits:
+      memlock:
+        soft: -1
+        hard: -1
+      nofile:
+        soft: 65536 
+        hard: 65536
+    volumes:
+      - ${DB_LOCATION}:/usr/share/opensearch/data:z
     ports:
-      - "8000:8000"
+      - 9200:9200
     networks:
       - shuffle
     restart: unless-stopped
-    volumes:
+  #docker-socket-proxy:
-      - ${DB_LOCATION}:/etc/shuffle
+  #  image: tecnativa/docker-socket-proxy
+  #  privileged: true
+  #  environment:
+  #    - SERVICES=1
+  #    - TASKS=1
+  #    - NETWORKS=1
+  #    - NODES=1
+  #    - BUILD=1
+  #    - IMAGES=1
+  #    - GRPC=1
+  #    - CONTAINERS=1
+  #    - PLUGINS=1
+  #    - SYSTEM=1
+  #    - VOLUMES=1
+  #    - INFO=1
+  #    - DISTRIBUTION=1
+  #    - POST=1
+  #    - AUTH=1
+  #    - SECRETS=1
+  #  volumes:
+  #    - /var/run/docker.sock:/var/run/docker.sock
+  #  networks:
+  #    - shuffle
 networks:
   shuffle:
     driver: bridge
+    #driver: overlay 

frontend/Dockerfile

@@ -1,5 +1,5 @@
 # Build environment
-FROM node as builder
+FROM node:14 as builder
 
 RUN mkdir /usr/src/app
 
@@ -8,7 +8,7 @@ ENV PATH /usr/src/app/node_modules/.bin:$PATH
 
 COPY package.json /usr/src/app/package.json
 
-#RUN npm install --verbose
+RUN yarn config set "strict-ssl" false -g
 RUN yarn install 
 
 # copy only required files to not trigger rebuilding every time
@@ -18,14 +18,11 @@ COPY ./src /usr/src/app/src/
 COPY ./*.sh /usr/src/app/
 COPY ./*.json /usr/src/app/
 
-# There were issues with the webpack installer from package.json
 RUN rm -rf /usr/src/app/node_modules/webpack
-#RUN yarn add webpack@4.42.0
-
 RUN yarn build
 
 # Production environment
-FROM nginx:latest
+FROM nginx:1.21.5
 
 RUN mkdir -p /usr/share/nginx/html/build
 RUN mkdir -p /usr/share/nginx/html/css

frontend/README.md

@@ -1,4 +1,5 @@
 # Certificate:
+
 Creating a localhost certificate:
 
 ```

frontend/confd/templates/nginx.conf

@@ -28,6 +28,28 @@ http {
 	server {
 		listen         80;
 		server_name    "localhost";
+
+		#location /static/js/* {
+		#	# avoid clickjacking
+		#	add_header X-Frame-Options DENY;
+		#	add_header X-Content-Type-Options nosniff;
+		#	add_header ;
+		#	# block MIME sniffing
+
+		#	# security headers
+		#	add_header X-XSS-Protection "1; mode=block";
+		#	# add_header Content-Security-Policy "default-src 'self'";
+		#	add_header Referrer-Policy "no-referrer";
+		#	server_tokens off;
+
+		#	root /usr/share/nginx/html;
+		#	gzip_static on;
+		#	expires 1y;
+		#	add_header Cache-Control public;
+		#	add_header ETag "";
+		#	try_files $uri /index.html;
+		#}
+
 		location / {
 			# avoid clickjacking
 			add_header X-Frame-Options DENY;

frontend/package.json

@@ -1,42 +1,54 @@
 {
   "name": "shuffler",
   "homepage": "https://shuffler.io",
-  "version": "0.8.53",
+  "version": "0.9.61",
   "private": true,
   "dependencies": {
+    "@babel/core": "^7.15.8",
+    "@emotion/is-prop-valid": "^1.1.1",
+    "@emotion/react": "^11.7.0",
+    "@emotion/styled": "^11.6.0",
     "@material-ui/core": "^4.5.2",
+    "@material-ui/data-grid": "^4.0.0-alpha.22",
     "@material-ui/icons": "^4.5.1",
+    "@material-ui/lab": "^4.0.0-alpha.58",
     "@material-ui/styles": "^4.5.2",
+    "@material-ui/utils": "^4.11.2",
+    "@metamask/detect-provider": "^1.2.0",
+    "@mui/icons-material": "^5.2.1",
+    "@mui/material": "^5.2.3",
+    "@uiw/react-codemirror": "^3.2.1",
     "@use-it/interval": "^1.0.0",
     "babel-eslint": "^10.1.0",
-    "class-transformer": "^0.3.1",
+    "class-transformer": "^0.4.0",
-    "create-react-app": "^2.0.3",
+    "create-react-app": "^4.0.3",
-    "cytoscape": "^3.11.0",
+    "cytoscape": "^3.15.1",
     "cytoscape-clipboard": "^2.2.1",
     "cytoscape-cxtmenu": "^3.1.1",
     "cytoscape-edgehandles": "^3.6.0",
-    "cytoscape-grid-guide": "~2.1.2",
+    "cytoscape-grid-guide": "~2.3.3",
     "cytoscape-node-html-label": "^1.1.5",
     "cytoscape-panzoom": "^2.5.2",
     "cytoscape-undo-redo": "^1.3.2",
-    "d3": "~4.10.0",
+    "d3": "^7.1.1",
     "dotenv": "^6.1.0",
     "downshift": "^3.3.5",
     "github-markdown-css": "^3.0.1",
     "import": "0.0.6",
     "interweave": "^11.2.0",
-    "material-icons": "^0.3.1",
+    "material-icons": "^0.7.7",
     "material-icons-react": "^1.0.4",
     "material-ui-chip-input": "^2.0.0-beta.2",
     "material-ui-nested-menu-item": "^1.0.2",
     "md5-file": "^4.0.0",
     "mdbreact": "^4.21.1",
-    "moment": "~2.20.1",
+    "moment": "^2.29.1",
     "react": "^16.14.0",
     "react-alert": "^5.5.0",
     "react-alert-template-basic": "^1.0.0",
+    "react-avatar-editor": "^11.1.0",
     "react-beforeunload": "^2.2.1",
-    "react-chartjs-2": "^2.8.0",
+    "react-chartjs-2": "^2.11.1",
     "react-cookie": "^4.0.1",
     "react-cytoscapejs": "^1.2.0",
     "react-device-detect": "^1.9.10",
@@ -50,27 +62,36 @@
     "react-markdown": "^4.2.2",
     "react-markdown-github": "^3.3.1",
     "react-powerhooks": "0.0.7",
-    "react-router": "^4.3.1",
+    "react-router": "6.2.1",
-    "react-router-dom": "^4.3.1",
+    "react-router-dom": "6.2.1",
-    "react-scripts": "^3.4.1",
+    "react-scripts": "^4.0.1",
+    "react-shepherd": "^3.3.6",
     "reactstrap": "^7.1.0",
+    "reaviz": "^12.1.0",
     "shellwords": "^0.1.1",
     "simplebar": "^4.2.3",
     "styled-components": "^4.4.0",
-    "webpack": "^4.42.0",
+    "webpack": "^4.44.2",
     "websocket": "^1.0.30",
     "yaml": "^1.7.2",
     "yamljs": "^0.3.0",
-    "zone.js": "~0.8.26"
+    "zone.js": "~0.11.4"
   },
   "scripts": {
-    "start": "set HTTPS=true&&react-scripts start",
+    "start": "react-scripts start",
     "build": "react-scripts build",
     "test": "react-scripts test",
-    "eject": "react-scripts eject"
+    "eject": "react-scripts eject",
+    "lint": "eslint 'src/**/*.{tsx,ts,js,jsx}'",
+    "lint_file": "eslint 'src/views/AngularWorkflow.jsx'"
   },
   "eslintConfig": {
-    "extends": "react-app"
+    "extends": "react-app",
+    "rules": {
+      "jsx-a11y/img-redundant-alt": "off",
+      "no-redeclare": "off",
+      "no-loop-func": "off"
+    }
   },
   "browserslist": [
     ">0.2%",
@@ -79,6 +100,7 @@
     "not op_mini all"
   ],
   "devDependencies": {
+    "prettier": "2.4.1",
     "promise-window": "^1.2.1"
   }
 }

frontend/public/images/btn_google_light_focus_ios.svg

@@ -0,0 +1,44 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<svg width="46px" height="46px" viewBox="0 0 46 46" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:sketch="http://www.bohemiancoding.com/sketch/ns">
+    <!-- Generator: Sketch 3.3.3 (12081) - http://www.bohemiancoding.com/sketch -->
+    <title>btn_google_light_focus_ios</title>
+    <desc>Created with Sketch.</desc>
+    <defs>
+        <filter x="-50%" y="-50%" width="200%" height="200%" filterUnits="objectBoundingBox" id="filter-1">
+            <feOffset dx="0" dy="1" in="SourceAlpha" result="shadowOffsetOuter1"></feOffset>
+            <feGaussianBlur stdDeviation="0.5" in="shadowOffsetOuter1" result="shadowBlurOuter1"></feGaussianBlur>
+            <feColorMatrix values="0 0 0 0 0   0 0 0 0 0   0 0 0 0 0  0 0 0 0.168 0" in="shadowBlurOuter1" type="matrix" result="shadowMatrixOuter1"></feColorMatrix>
+            <feOffset dx="0" dy="0" in="SourceAlpha" result="shadowOffsetOuter2"></feOffset>
+            <feGaussianBlur stdDeviation="0.5" in="shadowOffsetOuter2" result="shadowBlurOuter2"></feGaussianBlur>
+            <feColorMatrix values="0 0 0 0 0   0 0 0 0 0   0 0 0 0 0  0 0 0 0.084 0" in="shadowBlurOuter2" type="matrix" result="shadowMatrixOuter2"></feColorMatrix>
+            <feMerge>
+                <feMergeNode in="shadowMatrixOuter1"></feMergeNode>
+                <feMergeNode in="shadowMatrixOuter2"></feMergeNode>
+                <feMergeNode in="SourceGraphic"></feMergeNode>
+            </feMerge>
+        </filter>
+        <rect id="path-2" x="0" y="0" width="40" height="40" rx="2"></rect>
+    </defs>
+    <g id="Google-Button" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd" sketch:type="MSPage">
+        <g id="9-PATCH" sketch:type="MSArtboardGroup" transform="translate(-668.000000, -160.000000)"></g>
+        <g id="btn_google_light_focus" sketch:type="MSArtboardGroup" transform="translate(-1.000000, -1.000000)">
+            <rect id="Rectangle-14" fill-opacity="0.3" fill="#4285F4" sketch:type="MSShapeGroup" x="1" y="1" width="46" height="46"></rect>
+            <g id="button" sketch:type="MSLayerGroup" transform="translate(4.000000, 4.000000)" filter="url(#filter-1)">
+                <g id="button-bg">
+                    <use fill="#FFFFFF" fill-rule="evenodd" sketch:type="MSShapeGroup" xlink:href="#path-2"></use>
+                    <use fill="none" xlink:href="#path-2"></use>
+                    <use fill="none" xlink:href="#path-2"></use>
+                    <use fill="none" xlink:href="#path-2"></use>
+                </g>
+            </g>
+            <g id="logo_googleg_48dp" sketch:type="MSLayerGroup" transform="translate(15.000000, 15.000000)">
+                <path d="M17.64,9.20454545 C17.64,8.56636364 17.5827273,7.95272727 17.4763636,7.36363636 L9,7.36363636 L9,10.845 L13.8436364,10.845 C13.635,11.97 13.0009091,12.9231818 12.0477273,13.5613636 L12.0477273,15.8195455 L14.9563636,15.8195455 C16.6581818,14.2527273 17.64,11.9454545 17.64,9.20454545 L17.64,9.20454545 Z" id="Shape" fill="#4285F4" sketch:type="MSShapeGroup"></path>
+                <path d="M9,18 C11.43,18 13.4672727,17.1940909 14.9563636,15.8195455 L12.0477273,13.5613636 C11.2418182,14.1013636 10.2109091,14.4204545 9,14.4204545 C6.65590909,14.4204545 4.67181818,12.8372727 3.96409091,10.71 L0.957272727,10.71 L0.957272727,13.0418182 C2.43818182,15.9831818 5.48181818,18 9,18 L9,18 Z" id="Shape" fill="#34A853" sketch:type="MSShapeGroup"></path>
+                <path d="M3.96409091,10.71 C3.78409091,10.17 3.68181818,9.59318182 3.68181818,9 C3.68181818,8.40681818 3.78409091,7.83 3.96409091,7.29 L3.96409091,4.95818182 L0.957272727,4.95818182 C0.347727273,6.17318182 0,7.54772727 0,9 C0,10.4522727 0.347727273,11.8268182 0.957272727,13.0418182 L3.96409091,10.71 L3.96409091,10.71 Z" id="Shape" fill="#FBBC05" sketch:type="MSShapeGroup"></path>
+                <path d="M9,3.57954545 C10.3213636,3.57954545 11.5077273,4.03363636 12.4404545,4.92545455 L15.0218182,2.34409091 C13.4631818,0.891818182 11.4259091,0 9,0 C5.48181818,0 2.43818182,2.01681818 0.957272727,4.95818182 L3.96409091,7.29 C4.67181818,5.16272727 6.65590909,3.57954545 9,3.57954545 L9,3.57954545 Z" id="Shape" fill="#EA4335" sketch:type="MSShapeGroup"></path>
+                <path d="M0,0 L18,0 L18,18 L0,18 L0,0 Z" id="Shape" sketch:type="MSShapeGroup"></path>
+            </g>
+            <g id="handles_square" sketch:type="MSLayerGroup"></g>
+        </g>
+    </g>
+</svg>

frontend/public/index.html

@@ -1,13 +1,15 @@
 <!DOCTYPE html>
 <html lang="en">
   <head>
-    <meta charset="utf-8">
+    <meta charset="utf-8" />
-    <link rel="shortcut icon" href="%PUBLIC_URL%/favicon.ico">
+    <link rel="shortcut icon" href="%PUBLIC_URL%/favicon.ico" />
-    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
+    <meta
-    <meta name="theme-color" content="#000000">
+      name="viewport"
-    <link rel="manifest" href="%PUBLIC_URL%/manifest.json">
+      content="width=device-width, initial-scale=1, shrink-to-fit=no"
+    />
+    <meta name="theme-color" content="#000000" />
+    <link rel="manifest" href="%PUBLIC_URL%/manifest.json" />
     <title>Shuffle</title>
-
   </head>
   <body>
     <div id="root"></div>

frontend/run.sh

@@ -1,11 +1,11 @@
 #!/bin/sh
 docker stop shuffle-frontend
 docker rm shuffle-frontend
-docker rmi frikky/shuffle:frontend
+#docker rmi ghcr.io/frikky/shuffle-frontend:nightly
 
 echo "Running build for website"
 #sudo npm run build
-docker build . -t frikky/shuffle:frontend
+docker build . -t ghcr.io/frikky/shuffle-frontend:nightly
 
 echo "Starting server"
 # Rerun build locally for it to update :)

frontend/src/App.jsx

@@ -1,170 +1,687 @@
-import React, { useState, useEffect } from 'react';
+import React, { useState, useEffect } from "react";
 
-import { Route } from 'react-router';
+//import { Route, Routes } from "react-router";
-import { BrowserRouter } from 'react-router-dom';
+import { Route, Routes, BrowserRouter } from "react-router-dom";
-import { CookiesProvider } from 'react-cookie';
+import { CookiesProvider } from "react-cookie";
-import { removeCookies, useCookies } from 'react-cookie';
+import { removeCookies, useCookies } from "react-cookie";
 
-import EditSchedule from "./views/EditSchedule";
-import Schedules from "./views/Schedules";
-import Webhooks from "./views/Webhooks";
 import Workflows from "./views/Workflows";
+import GettingStarted from "./views/GettingStarted";
 import EditWebhook from "./views/EditWebhook";
 import AngularWorkflow from "./views/AngularWorkflow";
 
-import Header from './components/Header';
+import Header from "./components/Header";
-import Apps from './views/Apps';
+import theme from "./theme";
-import AppCreator from './views/AppCreator';
+import Apps from "./views/Apps";
-import Contact from './views/Contact';
+import AppCreator from "./views/AppCreator";
-import Oauth2 from './views/Oauth2';
 
-import Dashboard from "./views/Dashboard";
+import Dashboard from "./views/Dashboard.jsx";
 import AdminSetup from "./views/AdminSetup";
 import Admin from "./views/Admin";
 import Docs from "./views/Docs";
 import Introduction from "./views/Introduction";
+import SetAuthentication from "./views/SetAuthentication";
+import SetAuthenticationSSO from "./views/SetAuthenticationSSO";
 
 import LandingPageNew from "./views/LandingpageNew";
 import LoginPage from "./views/LoginPage";
 import SettingsPage from "./views/SettingsPage";
+import KeepAlive from "./views/KeepAlive.jsx";
 
-import { createMuiTheme, MuiThemeProvider } from '@material-ui/core/styles';
+import MyView from "./views/MyView";
 
+import { createMuiTheme, MuiThemeProvider } from "@material-ui/core/styles";
+
+import FrameworkWrapper from "./views/FrameworkWrapper.jsx";
+import ScrollToTop from "./components/ScrollToTop";
 import AlertTemplate from "./components/AlertTemplate";
-import { positions, Provider } from "react-alert";
+import { useAlert, positions, Provider } from "react-alert";
+import { isMobile } from "react-device-detect";
+
+import detectEthereumProvider from "@metamask/detect-provider";
+import Drift from "react-driftjs";
 
 // Production - backend proxy forwarding in nginx
-var globalUrl = window.location.origin
+var globalUrl = window.location.origin;
 
 // CORS used for testing purposes. Should only happen with specific port and http
-if (window.location.protocol == "http:" && window.location.port === "3000") {
+if (window.location.port === "3000") {
-	globalUrl = "http://localhost:5001"
+  globalUrl = "http://localhost:5001";
-	//globalUrl = "http://localhost:5002"
+  //globalUrl = "http://localhost:5002"
 }
 
-const theme = createMuiTheme({
+if (globalUrl.includes("githubpreview.dev")) {
-	palette: {
+	//globalUrl = globalUrl.replace("3000", "5001")
-		primary: {
+	globalUrl = "https://frikky-shuffle-5gvr4xx62w64-5001.githubpreview.dev"
-			main: "#f85a3e"
+}
-		},
+console.log("global: ", globalUrl)
-		secondary: {
-			main: '#e8eaf6',
-		},
-		surfaceColor: "#27292d",
-		inputColor: "#383B40"
-	},
-	typography: {
-		useNextVariants: true
-	},
-	overrides: {
-		MuiMenu: {
-			list: {
-				backgroundColor: "#383B40",
-			},
-		},
-	},
-});
 
-
-// FIXME - set client side cookies
 const App = (message, props) => {
-	const [userdata, setUserData] = useState({});
-	//const [homePage, ] = useState(true);
-	const [cookies, setCookie, removeCookie] = useCookies([]);
-	const [isLoggedIn, setIsLoggedIn] = useState(false);
-	const [dataset, setDataset] = useState(false);
-	const [isLoaded, setIsLoaded] = useState(false);
 
-	useEffect(() => {
+  const [userdata, setUserData] = useState({});
-		if (dataset === false) {
+  const [notifications, setNotifications] = useState([])
-			checkLogin()
+  const [cookies, setCookie, removeCookie] = useCookies([])
-			setDataset(true)
+  const [isLoggedIn, setIsLoggedIn] = useState(false)
-		}
+  const [dataset, setDataset] = useState(false)
-	})
+  const [isLoaded, setIsLoaded] = useState(false)
+  const [curpath, setCurpath] = useState(
+    typeof window === "undefined" || window.location === undefined
+      ? ""
+      : window.location.pathname
+  )
 
-	if (isLoaded && !isLoggedIn && (!window.location.pathname.startsWith("/login") && (!window.location.pathname.startsWith("/docs") && (!window.location.pathname.startsWith("/adminsetup"))))) {
+
-		window.location = "/login"
+  useEffect(() => {
+    if (dataset === false) {
+      getUserNotifications();
+      checkLogin();
+      setDataset(true);
+    }
+  }, []);
+
+  if (
+    isLoaded &&
+    !isLoggedIn &&
+    !window.location.pathname.startsWith("/login") &&
+    !window.location.pathname.startsWith("/docs") &&
+    !window.location.pathname.startsWith("/support") &&
+    !window.location.pathname.startsWith("/detectionframework") &&
+    !window.location.pathname.startsWith("/appframework") &&
+    !window.location.pathname.startsWith("/adminsetup") &&
+    !window.location.pathname.startsWith("/usecases")
+  ) {
+    window.location = "/login";
+  }
+
+  const getUserNotifications = () => {
+    fetch(`${globalUrl}/api/v1/users/notifications`, {
+      credentials: "include",
+      headers: {
+        "Content-Type": "application/json",
+      },
+			cors: "cors",
+    })
+      .then((response) => response.json())
+      .then((responseJson) => {
+        if (
+          responseJson.success === true &&
+          responseJson.notifications !== null &&
+          responseJson.notifications !== undefined &&
+          responseJson.notifications.length > 0
+        ) {
+          //console.log("RESP: ", responseJson)
+          setNotifications(responseJson.notifications);
+        }
+      })
+      .catch((error) => {
+        console.log("Failed getting notifications for user: ", error);
+      });
+  };
+
+  const checkLogin = () => {
+    var baseurl = globalUrl;
+    fetch(`${globalUrl}/api/v1/getinfo`, {
+      credentials: "include",
+      headers: {
+        "Content-Type": "application/json",
+      },
+    })
+      .then((response) => response.json())
+      .then((responseJson) => {
+        var userInfo = {};
+        if (responseJson.success === true) {
+          console.log(responseJson);
+
+          userInfo = responseJson;
+          setIsLoggedIn(true);
+          //console.log("Cookies: ", cookies)
+          // Updating cookie every request
+          for (var key in responseJson["cookies"]) {
+            setCookie(
+              responseJson["cookies"][key].key,
+              responseJson["cookies"][key].value,
+              { path: "/" }
+            );
+          }
+        }
+
+        // Handling Ethereum update
+				{/*
+        detectEthereumProvider().then((provider) => {
+          if (
+            provider &&
+            userInfo.eth_info !== undefined &&
+            userInfo.eth_info !== null
+          ) {
+            if (
+              userInfo.eth_info.account !== undefined &&
+              userInfo.eth_info.account !== null &&
+              userInfo.eth_info.account.length === 0
+            ) {
+              userInfo.eth_info = {};
+              var method = "eth_requestAccounts";
+              var params = [];
+              provider
+                .request({
+                  method: method,
+                  params,
+                })
+                .then((result) => {
+                  if (
+                    result !== undefined &&
+                    result !== null &&
+                    result.length > 0
+                  ) {
+                    userInfo.eth_info.account = result[0];
+
+                    // Getting and setting balance for the current user
+                    method = "eth_getBalance";
+                    params = [userInfo.eth_info.account, "latest"];
+                    provider
+                      .request({
+                        method: method,
+                        params,
+                      })
+                      .then((result) => {
+                        if (
+                          result !== undefined &&
+                          result !== null &&
+                          result.length > 0
+                        ) {
+                          userInfo.parsed_balance =
+                            result / 1000000000000000000;
+                        } else {
+                          alert.error("Couldn't find balance: ", result);
+                        }
+                        // The result varies by RPC method.
+                        // For example, this method will return a transaction hash hexadecimal string on success.
+                      })
+                      .catch((error) => {
+                        // If the request fails, the Promise will reject with an error.
+                        alert.error(
+                          "Failed getting info from ethereum API: " + error
+                        );
+                      });
+                  } else {
+                    alert.error("Couldn't find any user: ", result);
+                  }
+                })
+                .catch((error) => {
+                  // If the request fails, the Promise will reject with an error.
+                  alert.error(
+                    "Failed getting info from ethereum API: " + error
+                  );
+                });
+            }
+
+            // Register hooks here
+            provider.on("message", (event) => {
+              alert.info("Message from MetaMask: ", event);
+            });
+
+            provider.on("chainChanged", (chainId) => {
+              console.log("Changed chain to: ", chainId);
+
+              method = "eth_getBalance";
+              params = [userInfo.eth_info.account, "latest"];
+              provider
+                .request({
+                  method: method,
+                  params,
+                })
+                .then((result) => {
+                  console.log("Got result: ", result);
+                  if (result !== undefined && result !== null) {
+                    userInfo.eth_info.balance = result;
+                    userInfo.eth_info.parsed_balance =
+                      result / 1000000000000000000;
+                    console.log("INFO: ", userInfo);
+                    setUserData(userInfo);
+                  } else {
+                    alert.error("Couldn't find balance: ", result);
+                  }
+                })
+                .catch((error) => {
+                  // If the request fails, the Promise will reject with an error.
+                  alert.error(
+                    "Failed getting info from ethereum API: " + error
+                  );
+                });
+            });
+          }
+        });
+
+        if (
+          userInfo.eth_info !== undefined &&
+          userInfo.eth_info.balance !== undefined
+        ) {
+          //console.log(userInfo.eth_info.balance)
+          userInfo.eth_info.parsed_balance =
+            userInfo.eth_info.balance / 1000000000000000000;
+        }
+				*/}
+
+        //console.log("USER: ", userInfo)
+        setUserData(userInfo);
+        setIsLoaded(true);
+      })
+      .catch((error) => {
+        setIsLoaded(true);
+      });
+  };
+
+  // Dumb for content load (per now), but good for making the site not suddenly reload parts (ajax thingies)
+
+  const options = {
+    timeout: 9000,
+    position: positions.BOTTOM_LEFT,
+  };
+
+	const handleFirstInteraction = (event) => {
+		console.log("First interaction: ", event)
 	}
 
-	const checkLogin = () => {
+  const includedData =
-		var baseurl = globalUrl
+    window.location.pathname === "/home" ||
-		fetch(baseurl + "/api/v1/users/getinfo", {
+    window.location.pathname === "/features" ? (
-			credentials: "include",
+      <div>
-			headers: {
+				<Routes>
-				'Content-Type': 'application/json',
+					<Route
-			},
+						exact
-		})
+						path="/home"
-		.then(response => response.json())
+						render={(props) => <LandingPageNew isLoaded={isLoaded} {...props} />}
-		.then(responseJson => {
+					/>
-			if (responseJson.success === true) {
+				</Routes>
-				//console.log(responseJson.success)
+      </div>
-				setUserData(responseJson)
+    ) : (
-				setIsLoggedIn(true)
+      <div
-				//console.log("Cookies: ", cookies)
+        style={{
-
+          backgroundColor: "#1F2023",
-				// Updating cookie every request
+          color: "rgba(255, 255, 255, 0.65)",
-				for (var key in responseJson["cookies"]) {
+          minHeight: "100vh",
-					setCookie(responseJson["cookies"][key].key, responseJson["cookies"][key].value, { path: "/" })
+        }}
+      >
+        <ScrollToTop
+          getUserNotifications={getUserNotifications}
+          setCurpath={setCurpath}
+        />
+				{!isLoaded ? null : 
+					userdata.chat_disabled === true ? null : 
+						<Drift 
+							appId="zfk9i7w3yizf" 
+							attributes={{
+								name: userdata.username === undefined || userdata.username === null ? "OSS user" : `${userdata.username} - OSS`,
+							}}
+							eventHandlers={[
+								{ 
+									event: "conversation:firstInteraction", 
+									function: handleFirstInteraction 
+								},
+							]}
+						/>
+					
 				}
-			}
+        <Header
-			setIsLoaded(true)
+          notifications={notifications}
-		})
+          setNotifications={setNotifications}
-		.catch(error => {
+          checkLogin={checkLogin}
-			setIsLoaded(true)
+          cookies={cookies}
-		});
+          removeCookie={removeCookie}
-	}
+          isLoaded={isLoaded}
+          globalUrl={globalUrl}
+          setIsLoggedIn={setIsLoggedIn}
+          isLoggedIn={isLoggedIn}
+          userdata={userdata}
+          {...props}
+        />
+        <div style={{ height: 60 }} />
+				<Routes>
+        	<Route
+        	  exact
+        	  path="/login"
+        	  element={
+        	    <LoginPage
+        	      isLoggedIn={isLoggedIn}
+        	      setIsLoggedIn={setIsLoggedIn}
+        	      register={true}
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      setCookie={setCookie}
+        	      cookies={cookies}
+        	      checkLogin={checkLogin}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/admin"
+        	  element={
+        	    <Admin
+        	      userdata={userdata}
+        	      isLoggedIn={isLoggedIn}
+        	      setIsLoggedIn={setIsLoggedIn}
+        	      register={true}
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      setCookie={setCookie}
+        	      cookies={cookies}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/admin/:key"
+        	  element={
+        	    <Admin
+        	      isLoggedIn={isLoggedIn}
+        	      setIsLoggedIn={setIsLoggedIn}
+        	      register={true}
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      setCookie={setCookie}
+        	      cookies={cookies}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	{userdata.id !== undefined ? (
+        	  <Route
+        	    exact
+        	    path="/settings"
+        	    element={
+        	      <SettingsPage
+        	        isLoaded={isLoaded}
+        	        setUserData={setUserData}
+        	        userdata={userdata}
+        	        globalUrl={globalUrl}
+        	        {...props}
+        	      />
+        	    }
+        	  />
+        	) : null}
+        	<Route
+        	  exact
+        	  path="/AdminSetup"
+        	  element={
+        	    <AdminSetup
+        	      isLoaded={isLoaded}
+        	      userdata={userdata}
+        	      globalUrl={globalUrl}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/detectionframework"
+        	  element={
+        	    <FrameworkWrapper
+								selectedOption={"Draw"}
+								showOptions={false}
 
-	// Dumb for content load (per now), but good for making the site not suddenly reload parts (ajax thingies)
+        	      isLoaded={isLoaded}
+        	      isLoggedIn={isLoggedIn}
+        	      globalUrl={globalUrl}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/app"
+        	  element={
+        	    <FrameworkWrapper
+								selectedOption={"Draw"}
+								showOptions={false}
 
-	const options = {
+        	      isLoaded={isLoaded}
-		timeout: 9000,
+        	      isLoggedIn={isLoggedIn}
-		position: positions.BOTTOM_LEFT,
+        	      globalUrl={globalUrl}
-	};
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/usecases"
+        	  element={
+        	    <Dashboard
+        	      isLoaded={isLoaded}
+        	      isLoggedIn={isLoggedIn}
+        	      globalUrl={globalUrl}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/apps/new"
+        	  element={
+        	    <AppCreator
+        	      isLoaded={isLoaded}
+        	      isLoggedIn={isLoggedIn}
+        	      globalUrl={globalUrl}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/apps"
+        	  element={
+								<Apps
+									isLoaded={isLoaded}
+									isLoggedIn={isLoggedIn}
+									globalUrl={globalUrl}
+									userdata={userdata}
+									{...props}
+        	    	/>
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/apps/edit/:appid"
+        	  element={
+        	    <AppCreator
+        	      isLoaded={isLoaded}
+        	      isLoggedIn={isLoggedIn}
+        	      globalUrl={globalUrl}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/workflows"
+        	  element={
+        	    <Workflows
+        	      cookies={cookies}
+        	      removeCookie={removeCookie}
+        	      isLoaded={isLoaded}
+        	      isLoggedIn={isLoggedIn}
+        	      globalUrl={globalUrl}
+        	      cookies={cookies}
+        	      userdata={userdata}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/getting-started"
+        	  element={
+        	    <GettingStarted
+        	      cookies={cookies}
+        	      removeCookie={removeCookie}
+        	      isLoaded={isLoaded}
+        	      isLoggedIn={isLoggedIn}
+        	      globalUrl={globalUrl}
+        	      cookies={cookies}
+        	      userdata={userdata}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/workflows/:key"
+        	  element={
+        	    <AngularWorkflow
+								alert={alert}
+        	      userdata={userdata}
+        	      globalUrl={globalUrl}
+        	      isLoaded={isLoaded}
+        	      isLoggedIn={isLoggedIn}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/docs/:key"
+        	  element={
+        	    <Docs
+        	      isMobile={isMobile}
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/docs"
+        	  element={
+							//navigate(`/docs/about`)
+        	    <Docs
+        	      isMobile={isMobile}
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/support"
+        	  element={
+							//navigate(`/docs/about`)
+        	    <Docs
+        	      isMobile={isMobile}
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/introduction"
+        	  element={
+        	    <Introduction
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/introduction/:key"
+        	  element={
+        	    <Introduction
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/set_authentication"
+        	  element={
+        	    <SetAuthentication
+        	      userdata={userdata}
+        	      isLoggedIn={isLoggedIn}
+        	      setIsLoggedIn={setIsLoggedIn}
+        	      register={true}
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      setCookie={setCookie}
+        	      cookies={cookies}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/login_sso"
+        	  element={
+        	    <SetAuthenticationSSO
+        	      userdata={userdata}
+        	      isLoggedIn={isLoggedIn}
+        	      setIsLoggedIn={setIsLoggedIn}
+        	      register={true}
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      setCookie={setCookie}
+        	      cookies={cookies}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/keepalive"
+        	  element={
+        	    <KeepAlive
+        	      isLoggedIn={isLoggedIn}
+        	      setIsLoggedIn={setIsLoggedIn}
+        	      register={true}
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      setCookie={setCookie}
+        	      cookies={cookies}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+        	<Route
+        	  exact
+        	  path="/"
+        	  element={
+        	    <LoginPage
+        	      isLoggedIn={isLoggedIn}
+        	      setIsLoggedIn={setIsLoggedIn}
+        	      register={true}
+        	      isLoaded={isLoaded}
+        	      globalUrl={globalUrl}
+        	      setCookie={setCookie}
+        	      cookies={cookies}
+        	      {...props}
+        	    />
+        	  }
+        	/>
+				</Routes>
+      </div>
+    );
 
-	const includedData = window.location.pathname === "/home" || window.location.pathname === "/features" ?
+  // <div style={{backgroundColor: "rgba(21, 32, 43, 1)", color: "#fffff", minHeight: "100vh"}}>
-		<div>
+  // backgroundColor: "#213243",
-			<Route exact path="/home" render={props => <LandingPageNew isLoaded={isLoaded} {...props} />} />
+  // This is a mess hahahah
-		</div> :
+  return (
-		<div style={{ backgroundColor: "#1F2023", color: "rgba(255, 255, 255, 0.65)", minHeight: "100vh" }}>
+    <MuiThemeProvider theme={theme}>
-			<Header cookies={cookies} removeCookie={removeCookie} isLoaded={isLoaded} globalUrl={globalUrl} setIsLoggedIn={setIsLoggedIn} isLoggedIn={isLoggedIn} userdata={userdata} {...props} />
+      <CookiesProvider>
-			<Route exact path="/oauth2" render={props => <Oauth2 isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
+        <BrowserRouter>
-			<Route exact path="/contact" render={props => <Contact isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
+          <Provider template={AlertTemplate} {...options}>
-			<Route exact path="/login" render={props => <LoginPage isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
+            {includedData}
-			<Route exact path="/admin" render={props => <Admin userdata={userdata} isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
+          </Provider>
-			<Route exact path="/admin/:key" render={props => <Admin isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
+        </BrowserRouter>
-			<Route exact path="/settings" render={props => <SettingsPage isLoaded={isLoaded} userdata={userdata} globalUrl={globalUrl} {...props} />} />
+      </CookiesProvider>
-			<Route exact path="/AdminSetup" render={props => <AdminSetup isLoaded={isLoaded} userdata={userdata} globalUrl={globalUrl} {...props} />} />
+    </MuiThemeProvider>
-			<Route exact path="/webhooks" render={props => <Webhooks isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
+  );
-			<Route exact path="/webhooks/:key" render={props => <EditWebhook isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
-			<Route exact path="/schedules" render={props => <Schedules globalUrl={globalUrl} {...props} />} />
-			<Route exact path="/dashboard" render={props => <Dashboard isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} {...props} />} />
-			<Route exact path="/apps" render={props => <Apps isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} userdata={userdata} {...props} />} />
-			<Route exact path="/apps/new" render={props => <AppCreator isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} {...props} />} />
-			<Route exact path="/apps/edit/:appid" render={props => <AppCreator isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} {...props} />} />
-			<Route exact path="/schedules/:key" render={props => <EditSchedule globalUrl={globalUrl} {...props} />} />
-			<Route exact path="/workflows" render={props => <Workflows cookies={cookies} removeCookie={removeCookie} isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} cookies={cookies} userdata={userdata} {...props} />} />
-			<Route exact path="/workflows/:key" render={props => <AngularWorkflow userdata={userdata} globalUrl={globalUrl} isLoaded={isLoaded} isLoggedIn={isLoggedIn} {...props} />} />
-			<Route exact path="/docs/:key" render={props => <Docs isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
-			<Route exact path="/docs" render={props => { window.location.pathname = "/docs/about" }} />
-			<Route exact path="/introduction" render={props => <Introduction isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
-			<Route exact path="/introduction/:key" render={props => <Introduction isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
-			<Route exact path="/" render={props => <LoginPage isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
-		</div>
-
-	// <div style={{backgroundColor: "rgba(21, 32, 43, 1)", color: "#fffff", minHeight: "100vh"}}>
-	// backgroundColor: "#213243",
-	// This is a mess hahahah
-	return (
-		<MuiThemeProvider theme={theme}>
-			<CookiesProvider>
-				<BrowserRouter>
-					<Provider template={AlertTemplate} {...options}>
-						{includedData}
-					</Provider>
-				</BrowserRouter>
-			</CookiesProvider>
-		</MuiThemeProvider>
-	);
 };
 
 export default App;

frontend/src/__test__/environmentdata.js

@@ -1,3 +1,6 @@
-const data = [{"name": "cloud", "type": "cloud"}, {"name": "onprem", "type": "onprem"}]
+const data = [
+  { name: "cloud", type: "cloud" },
+  { name: "onprem", type: "onprem" },
+];
 
-export default data; 
+export default data;

frontend/src/__test__/scheduledata.js

@@ -1,30 +1,29 @@
 const Data = {
-	"src": {
+  src: {
-		"name": "Get Tickets",
+    name: "Get Tickets",
-		"description": "Get tickets",
+    description: "Get tickets",
-		"outputparameters": [{
+    outputparameters: [
-			"name": "SymptomDescription",
+      {
-			"schema": {"type": "string"}},
+        name: "SymptomDescription",
-			{"name": "DetailedDescription",
+        schema: { type: "string" },
-			"schema": {"type": "string"}},
+      },
-			{"name": "EventSource",
+      { name: "DetailedDescription", schema: { type: "string" } },
-			"schema": {"type": "string"}
+      { name: "EventSource", schema: { type: "string" } },
-		}]
+    ],
-	},
+  },
-	"dst": {
+  dst: {
-		"name": "Create alert",
+    name: "Create alert",
-		"description": "Create alert in TheHive",
+    description: "Create alert in TheHive",
-		"inputparameters": [{
+    inputparameters: [
-			"name": "title",
+      {
-			"required": true,
+        name: "title",
-			"schema": {"type": "string"}},
+        required: true,
-			{"name": "description",
+        schema: { type: "string" },
-			"required": true,
+      },
-			"schema": {"type": "string"}},
+      { name: "description", required: true, schema: { type: "string" } },
-			{"name": "source",
+      { name: "source", required: true, schema: { type: "string" } },
-			"required": true,
+    ],
-			"schema": {"type": "string"}
+  },
-	}]}
 };
 
 export default Data;

frontend/src/__test__/webhookdata.js

@@ -1,15 +1,15 @@
 const data = {
-	"id":"8ccf0bec1fde018771ab685d2a40bd52",
+  id: "8ccf0bec1fde018771ab685d2a40bd52",
-	"info":{
+  info: {
-		"url":"",
+    url: "",
-		"name":"testing",
+    name: "testing",
-		"description":"wut"
+    description: "wut",
-	},
+  },
-	"transforms":{},
+  transforms: {},
-	"actions": {},
+  actions: {},
-	"type":"webhook",
+  type: "webhook",
-	"status":"uninitialized",
+  status: "uninitialized",
-	"running":false
+  running: false,
-}
+};
 
 export default data;

frontend/src/__test__/workflowdata.js

@@ -1,3 +1,169 @@
-const data = {"actions":[{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"70574332-da82-cf17-c723-75fa7b8493c2","is_valid":true,"label":"hello_world","environment":"onprem","name":"hello_world","parameters":null,"position":{"x":353.7438792397648,"y":260.6717930890377},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"30522433-56ed-53c3-575d-766e282e1d3e","is_valid":true,"label":"random_number","environment":"cloud","name":"random_number","parameters":null,"position":{"x":458.30040774503794,"y":104.27580103487651},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"5b7ac5b5-9514-02b9-ebe0-998c0843b104","is_valid":false,"label":"hello_world_2","environment":"onprem","name":"hello_world","parameters":null,"position":{"x":414.7256019053981,"y":-140.46450482659628},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"7e6e7a19-4636-cebc-91c4-052a3769a18b","is_valid":true,"label":"hello_world_3","environment":"cloud","name":"hello_world","parameters":null,"position":{"x":83.59752786243806,"y":50.232317715020734},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"edbf927d-5a00-2405-28ed-47982cdf5110","is_valid":true,"label":"hello_world_4","environment":"cloud","name":"hello_world","parameters":null,"position":{"x":-147.30681300186404,"y":89.16690830150289},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"4844a855-1e2b-669d-fc72-5f398321ac5d","is_valid":false,"label":"hello_world_5","environment":"onprem","name":"hello_world","parameters":null,"position":{"x":130.24982593523967,"y":233.8325632286361},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09","is_valid":true,"label":"hello_world_6","environment":"cloud","name":"hello_world","parameters":null,"position":{"x":83.551088005629,"y":-105.15867327274223},"priority":0},{"app_name":"hello_world","app_version":"1.0.0","errors":null,"id":"469d8c2b-52ac-e397-9a29-becccd04aed8","is_valid":true,"label":"hello_world_7","environment":"cloud","name":"hello_world","parameters":null,"position":{"x":314.4987657226086,"y":10.167183586257954},"priority":0}],"branches":[{"destination_id":"30522433-56ed-53c3-575d-766e282e1d3e","id":"4bcb9795-94e6-7d5f-2074-0d5b27784e0b","source_id":"70574332-da82-cf17-c723-75fa7b8493c2"},{"destination_id":"5b7ac5b5-9514-02b9-ebe0-998c0843b104","id":"fe0ab8e4-a535-61cd-3c09-8fd3d8e40769","source_id":"30522433-56ed-53c3-575d-766e282e1d3e"},{"destination_id":"469d8c2b-52ac-e397-9a29-becccd04aed8","id":"8b9ee9bc-b0ab-0bb6-af61-46d4594b2663","source_id":"30522433-56ed-53c3-575d-766e282e1d3e"},{"destination_id":"6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09","id":"c204d5ef-9cc1-d906-9988-86a624c57783","source_id":"469d8c2b-52ac-e397-9a29-becccd04aed8"},{"destination_id":"6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09","id":"1ffb3934-60ec-8f80-5cee-3ddc0a37fdb6","source_id":"5b7ac5b5-9514-02b9-ebe0-998c0843b104"},{"destination_id":"edbf927d-5a00-2405-28ed-47982cdf5110","id":"9c7fb048-9d0d-cb84-9ba0-be729af9b4d1","source_id":"6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09"},{"destination_id":"edbf927d-5a00-2405-28ed-47982cdf5110","id":"e3ab104e-fc8b-3af5-8daa-bfa57bcf9690","source_id":"7e6e7a19-4636-cebc-91c4-052a3769a18b"},{"destination_id":"7e6e7a19-4636-cebc-91c4-052a3769a18b","id":"b6626081-22dd-3af3-b899-480f60d886ca","source_id":"30522433-56ed-53c3-575d-766e282e1d3e"},{"destination_id":"4844a855-1e2b-669d-fc72-5f398321ac5d","id":"4275cf97-0447-bbda-0c80-ab20d389de1a","source_id":"edbf927d-5a00-2405-28ed-47982cdf5110"}],"conditions":[],"triggers":[],"transforms":[],"description":"asd","id":"2f299808-0f1b-4ae0-97fc-ac17483dfcf7","id":"2f299808-0f1b-4ae0-97fc-ac17483dfcf7","is_valid":true,"name":"test2","start":"70574332-da82-cf17-c723-75fa7b8493c2","owner":{"username":"","id":"","orgs":""},"execution_org":{"name":"","org":"","users":null,"id":""},"workflow_variables":null}
+const data = {
+  actions: [
+    {
+      app_name: "hello_world",
+      app_version: "1.0.0",
+      errors: null,
+      id: "70574332-da82-cf17-c723-75fa7b8493c2",
+      is_valid: true,
+      label: "hello_world",
+      environment: "onprem",
+      name: "hello_world",
+      parameters: null,
+      position: { x: 353.7438792397648, y: 260.6717930890377 },
+      priority: 0,
+    },
+    {
+      app_name: "hello_world",
+      app_version: "1.0.0",
+      errors: null,
+      id: "30522433-56ed-53c3-575d-766e282e1d3e",
+      is_valid: true,
+      label: "random_number",
+      environment: "cloud",
+      name: "random_number",
+      parameters: null,
+      position: { x: 458.30040774503794, y: 104.27580103487651 },
+      priority: 0,
+    },
+    {
+      app_name: "hello_world",
+      app_version: "1.0.0",
+      errors: null,
+      id: "5b7ac5b5-9514-02b9-ebe0-998c0843b104",
+      is_valid: false,
+      label: "hello_world_2",
+      environment: "onprem",
+      name: "hello_world",
+      parameters: null,
+      position: { x: 414.7256019053981, y: -140.46450482659628 },
+      priority: 0,
+    },
+    {
+      app_name: "hello_world",
+      app_version: "1.0.0",
+      errors: null,
+      id: "7e6e7a19-4636-cebc-91c4-052a3769a18b",
+      is_valid: true,
+      label: "hello_world_3",
+      environment: "cloud",
+      name: "hello_world",
+      parameters: null,
+      position: { x: 83.59752786243806, y: 50.232317715020734 },
+      priority: 0,
+    },
+    {
+      app_name: "hello_world",
+      app_version: "1.0.0",
+      errors: null,
+      id: "edbf927d-5a00-2405-28ed-47982cdf5110",
+      is_valid: true,
+      label: "hello_world_4",
+      environment: "cloud",
+      name: "hello_world",
+      parameters: null,
+      position: { x: -147.30681300186404, y: 89.16690830150289 },
+      priority: 0,
+    },
+    {
+      app_name: "hello_world",
+      app_version: "1.0.0",
+      errors: null,
+      id: "4844a855-1e2b-669d-fc72-5f398321ac5d",
+      is_valid: false,
+      label: "hello_world_5",
+      environment: "onprem",
+      name: "hello_world",
+      parameters: null,
+      position: { x: 130.24982593523967, y: 233.8325632286361 },
+      priority: 0,
+    },
+    {
+      app_name: "hello_world",
+      app_version: "1.0.0",
+      errors: null,
+      id: "6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09",
+      is_valid: true,
+      label: "hello_world_6",
+      environment: "cloud",
+      name: "hello_world",
+      parameters: null,
+      position: { x: 83.551088005629, y: -105.15867327274223 },
+      priority: 0,
+    },
+    {
+      app_name: "hello_world",
+      app_version: "1.0.0",
+      errors: null,
+      id: "469d8c2b-52ac-e397-9a29-becccd04aed8",
+      is_valid: true,
+      label: "hello_world_7",
+      environment: "cloud",
+      name: "hello_world",
+      parameters: null,
+      position: { x: 314.4987657226086, y: 10.167183586257954 },
+      priority: 0,
+    },
+  ],
+  branches: [
+    {
+      destination_id: "30522433-56ed-53c3-575d-766e282e1d3e",
+      id: "4bcb9795-94e6-7d5f-2074-0d5b27784e0b",
+      source_id: "70574332-da82-cf17-c723-75fa7b8493c2",
+    },
+    {
+      destination_id: "5b7ac5b5-9514-02b9-ebe0-998c0843b104",
+      id: "fe0ab8e4-a535-61cd-3c09-8fd3d8e40769",
+      source_id: "30522433-56ed-53c3-575d-766e282e1d3e",
+    },
+    {
+      destination_id: "469d8c2b-52ac-e397-9a29-becccd04aed8",
+      id: "8b9ee9bc-b0ab-0bb6-af61-46d4594b2663",
+      source_id: "30522433-56ed-53c3-575d-766e282e1d3e",
+    },
+    {
+      destination_id: "6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09",
+      id: "c204d5ef-9cc1-d906-9988-86a624c57783",
+      source_id: "469d8c2b-52ac-e397-9a29-becccd04aed8",
+    },
+    {
+      destination_id: "6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09",
+      id: "1ffb3934-60ec-8f80-5cee-3ddc0a37fdb6",
+      source_id: "5b7ac5b5-9514-02b9-ebe0-998c0843b104",
+    },
+    {
+      destination_id: "edbf927d-5a00-2405-28ed-47982cdf5110",
+      id: "9c7fb048-9d0d-cb84-9ba0-be729af9b4d1",
+      source_id: "6d1d3f8a-1ac9-3db4-0e0f-2fe32e9d3c09",
+    },
+    {
+      destination_id: "edbf927d-5a00-2405-28ed-47982cdf5110",
+      id: "e3ab104e-fc8b-3af5-8daa-bfa57bcf9690",
+      source_id: "7e6e7a19-4636-cebc-91c4-052a3769a18b",
+    },
+    {
+      destination_id: "7e6e7a19-4636-cebc-91c4-052a3769a18b",
+      id: "b6626081-22dd-3af3-b899-480f60d886ca",
+      source_id: "30522433-56ed-53c3-575d-766e282e1d3e",
+    },
+    {
+      destination_id: "4844a855-1e2b-669d-fc72-5f398321ac5d",
+      id: "4275cf97-0447-bbda-0c80-ab20d389de1a",
+      source_id: "edbf927d-5a00-2405-28ed-47982cdf5110",
+    },
+  ],
+  conditions: [],
+  triggers: [],
+  transforms: [],
+  description: "asd",
+  id: "2f299808-0f1b-4ae0-97fc-ac17483dfcf7",
+  id: "2f299808-0f1b-4ae0-97fc-ac17483dfcf7",
+  is_valid: true,
+  name: "test2",
+  start: "70574332-da82-cf17-c723-75fa7b8493c2",
+  owner: { username: "", id: "", orgs: "" },
+  execution_org: { name: "", org: "", users: null, id: "" },
+  workflow_variables: null,
+};
 
-export default data; 
+export default data;

frontend/src/charts.js

@@ -23,7 +23,7 @@
 let chart1_2_options = {
   maintainAspectRatio: false,
   legend: {
-    display: false
+    display: false,
   },
   tooltips: {
     backgroundColor: "#f5f5f5",
@@ -33,7 +33,7 @@ let chart1_2_options = {
     xPadding: 12,
     mode: "nearest",
     intersect: 0,
-    position: "nearest"
+    position: "nearest",
   },
   responsive: true,
   scales: {
@@ -43,15 +43,15 @@ let chart1_2_options = {
         gridLines: {
           drawBorder: false,
           color: "rgba(29,140,248,0.0)",
-          zeroLineColor: "transparent"
+          zeroLineColor: "transparent",
         },
         ticks: {
           suggestedMin: 60,
           suggestedMax: 125,
           padding: 20,
-          fontColor: "#9a9a9a"
+          fontColor: "#9a9a9a",
-        }
+        },
-      }
+      },
     ],
     xAxes: [
       {
@@ -59,22 +59,22 @@ let chart1_2_options = {
         gridLines: {
           drawBorder: false,
           color: "rgba(29,140,248,0.1)",
-          zeroLineColor: "transparent"
+          zeroLineColor: "transparent",
         },
         ticks: {
           padding: 20,
-          fontColor: "#9a9a9a"
+          fontColor: "#9a9a9a",
-        }
+        },
-      }
+      },
-    ]
+    ],
-  }
+  },
 };
 
 // #########################################
 // // // used inside src/views/Dashboard.js
 // #########################################
 let chartExample1 = {
-  data1: canvas => {
+  data1: (canvas) => {
     let ctx = canvas.getContext("2d");
 
     let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -96,7 +96,7 @@ let chartExample1 = {
         "SEP",
         "OCT",
         "NOV",
-        "DEC"
+        "DEC",
       ],
       datasets: [
         {
@@ -114,12 +114,12 @@ let chartExample1 = {
           pointHoverRadius: 4,
           pointHoverBorderWidth: 15,
           pointRadius: 4,
-          data: [100, 70, 90, 70, 85, 60, 75, 60, 90, 80, 110, 100]
+          data: [100, 70, 90, 70, 85, 60, 75, 60, 90, 80, 110, 100],
-        }
+        },
-      ]
+      ],
     };
   },
-  data2: canvas => {
+  data2: (canvas) => {
     let ctx = canvas.getContext("2d");
 
     let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -141,7 +141,7 @@ let chartExample1 = {
         "SEP",
         "OCT",
         "NOV",
-        "DEC"
+        "DEC",
       ],
       datasets: [
         {
@@ -159,12 +159,12 @@ let chartExample1 = {
           pointHoverRadius: 4,
           pointHoverBorderWidth: 15,
           pointRadius: 4,
-          data: [80, 120, 105, 110, 95, 105, 90, 100, 80, 95, 70, 120]
+          data: [80, 120, 105, 110, 95, 105, 90, 100, 80, 95, 70, 120],
-        }
+        },
-      ]
+      ],
     };
   },
-  data3: canvas => {
+  data3: (canvas) => {
     let ctx = canvas.getContext("2d");
 
     let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -186,7 +186,7 @@ let chartExample1 = {
         "SEP",
         "OCT",
         "NOV",
-        "DEC"
+        "DEC",
       ],
       datasets: [
         {
@@ -204,19 +204,19 @@ let chartExample1 = {
           pointHoverRadius: 4,
           pointHoverBorderWidth: 15,
           pointRadius: 4,
-          data: [60, 80, 65, 130, 80, 105, 90, 130, 70, 115, 60, 130]
+          data: [60, 80, 65, 130, 80, 105, 90, 130, 70, 115, 60, 130],
-        }
+        },
-      ]
+      ],
     };
   },
-  options: chart1_2_options
+  options: chart1_2_options,
 };
 
 // #########################################
 // // // used inside src/views/Dashboard.js
 // #########################################
 let chartExample2 = {
-  data: canvas => {
+  data: (canvas) => {
     let ctx = canvas.getContext("2d");
 
     let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -243,19 +243,19 @@ let chartExample2 = {
           pointHoverRadius: 4,
           pointHoverBorderWidth: 15,
           pointRadius: 4,
-          data: [80, 100, 70, 80, 120, 80]
+          data: [80, 100, 70, 80, 120, 80],
-        }
+        },
-      ]
+      ],
     };
   },
-  options: chart1_2_options
+  options: chart1_2_options,
 };
 
 // #########################################
 // // // used inside src/views/Dashboard.js
 // #########################################
 let chartExample3 = {
-  data: canvas => {
+  data: (canvas) => {
     let ctx = canvas.getContext("2d");
 
     let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -276,15 +276,15 @@ let chartExample3 = {
           borderWidth: 2,
           borderDash: [],
           borderDashOffset: 0.0,
-          data: [53, 20, 10, 80, 100, 45]
+          data: [53, 20, 10, 80, 100, 45],
-        }
+        },
-      ]
+      ],
     };
   },
   options: {
     maintainAspectRatio: false,
     legend: {
-      display: false
+      display: false,
     },
     tooltips: {
       backgroundColor: "#f5f5f5",
@@ -294,7 +294,7 @@ let chartExample3 = {
       xPadding: 12,
       mode: "nearest",
       intersect: 0,
-      position: "nearest"
+      position: "nearest",
     },
     responsive: true,
     scales: {
@@ -303,38 +303,38 @@ let chartExample3 = {
           gridLines: {
             drawBorder: false,
             color: "rgba(225,78,202,0.1)",
-            zeroLineColor: "transparent"
+            zeroLineColor: "transparent",
           },
           ticks: {
             suggestedMin: 60,
             suggestedMax: 120,
             padding: 20,
-            fontColor: "#9e9e9e"
+            fontColor: "#9e9e9e",
-          }
+          },
-        }
+        },
       ],
       xAxes: [
         {
           gridLines: {
             drawBorder: false,
             color: "rgba(225,78,202,0.1)",
-            zeroLineColor: "transparent"
+            zeroLineColor: "transparent",
           },
           ticks: {
             padding: 20,
-            fontColor: "#9e9e9e"
+            fontColor: "#9e9e9e",
-          }
+          },
-        }
+        },
-      ]
+      ],
-    }
+    },
-  }
+  },
 };
 
 // #########################################
 // // // used inside src/views/Dashboard.js
 // #########################################
 const chartExample4 = {
-  data: canvas => {
+  data: (canvas) => {
     let ctx = canvas.getContext("2d");
 
     let gradientStroke = ctx.createLinearGradient(0, 230, 0, 50);
@@ -361,15 +361,15 @@ const chartExample4 = {
           pointHoverRadius: 4,
           pointHoverBorderWidth: 15,
           pointRadius: 4,
-          data: [90, 27, 60, 12, 80]
+          data: [90, 27, 60, 12, 80],
-        }
+        },
-      ]
+      ],
     };
   },
   options: {
     maintainAspectRatio: false,
     legend: {
-      display: false
+      display: false,
     },
 
     tooltips: {
@@ -380,7 +380,7 @@ const chartExample4 = {
       xPadding: 12,
       mode: "nearest",
       intersect: 0,
-      position: "nearest"
+      position: "nearest",
     },
     responsive: true,
     scales: {
@@ -390,15 +390,15 @@ const chartExample4 = {
           gridLines: {
             drawBorder: false,
             color: "rgba(29,140,248,0.0)",
-            zeroLineColor: "transparent"
+            zeroLineColor: "transparent",
           },
           ticks: {
             suggestedMin: 50,
             suggestedMax: 125,
             padding: 20,
-            fontColor: "#9e9e9e"
+            fontColor: "#9e9e9e",
-          }
+          },
-        }
+        },
       ],
 
       xAxes: [
@@ -407,21 +407,21 @@ const chartExample4 = {
           gridLines: {
             drawBorder: false,
             color: "rgba(0,242,195,0.1)",
-            zeroLineColor: "transparent"
+            zeroLineColor: "transparent",
           },
           ticks: {
             padding: 20,
-            fontColor: "#9e9e9e"
+            fontColor: "#9e9e9e",
-          }
+          },
-        }
+        },
-      ]
+      ],
-    }
+    },
-  }
+  },
 };
 
 module.exports = {
   chartExample1, // in src/views/Dashboard.js
   chartExample2, // in src/views/Dashboard.js
   chartExample3, // in src/views/Dashboard.js
-  chartExample4 // in src/views/Dashboard.js
+  chartExample4, // in src/views/Dashboard.js
 };

frontend/src/components/AlertPopup.js

@@ -1,26 +1,19 @@
-import React, { useEffect} from 'react';
+import React, { useEffect } from "react";
 
 const Popup = (props) => {
-  	const { data } = props;
+  const { data } = props;
 
-	const popupStyle = {
+  const popupStyle = {
-		position: "fixed",
+    position: "fixed",
-		width: "300px",
+    width: "300px",
-		height: "50px",
+    height: "50px",
-		backgroundColor: "black",
+    backgroundColor: "black",
-		color: "white",
+    color: "white",
-	}
+  };
 
-	const popupData = 
+  const popupData = <div>HEY</div>;
-		<div>
-			HEY
-		</div>
 
-	return (
+  return <div>{popupData}</div>;
-		<div>
+};
-			{popupData}
-		</div>
-	)
-}
 
-export default Popup
+export default Popup;

frontend/src/components/AlertTemplate.js

@@ -1,45 +1,48 @@
-import React from 'react'
+import React from "react";
-import InfoIcon from '@material-ui/icons/Info';
+import InfoIcon from "@material-ui/icons/Info";
-import CheckIcon from '@material-ui/icons/Check';
+import CheckIcon from "@material-ui/icons/Check";
-import ErrorOutlineIcon from '@material-ui/icons/ErrorOutline';
+import ErrorOutlineIcon from "@material-ui/icons/ErrorOutline";
-import CloseIcon from '@material-ui/icons/Close';
+import CloseIcon from "@material-ui/icons/Close";
-import Typography from '@material-ui/core/Typography';
+import Typography from "@material-ui/core/Typography";
 
 const alertStyle = {
-  backgroundColor: 'rgba(0,0,0,0.9)',
+  backgroundColor: "rgba(0,0,0,0.9)",
-  color: 'white',
+  color: "white",
   padding: 15,
-  textTransform: 'uppercase',
+  textTransform: "uppercase",
-  borderRadius: '3px',
+  borderRadius: "3px",
-  display: 'flex',
+  display: "flex",
-  justifyContent: 'space-between',
+  justifyContent: "space-between",
-  alignItems: 'center',
+  alignItems: "center",
-  boxShadow: '0px 2px 2px 2px rgba(0, 0, 0, 0.03)',
+  boxShadow: "0px 2px 2px 2px rgba(0, 0, 0, 0.03)",
-  width: 400,
+  width: 300,
-  boxSizing: 'border-box',
+  boxSizing: "border-box",
-	zIndex: 100001,
+  zIndex: 100001,
-}
+  overflow: "hidden",
+};
 
 const buttonStyle = {
-  marginLeft: '20px',
+  marginLeft: "20px",
-  border: 'none',
+  border: "none",
-  backgroundColor: 'transparent',
+  backgroundColor: "transparent",
-  cursor: 'pointer',
+  cursor: "pointer",
-  color: '#FFFFFF'
+  color: "#FFFFFF",
-}
+};
 
 const AlertTemplate = ({ message, options, style, close }) => {
   return (
     <div style={{ ...alertStyle, ...style }}>
-      {options.type === 'info' && <InfoIcon style={{color: "white"}} />}
+      {options.type === "info" && <InfoIcon style={{ color: "white" }} />}
-      {options.type === 'success' && <CheckIcon style={{color: "green", }}/>}
+      {options.type === "success" && <CheckIcon style={{ color: "green" }} />}
-      {options.type === 'error' && <ErrorOutlineIcon style={{color: "red"}} />}
+      {options.type === "error" && (
-      <Typography style={{marginLeft: 15, flex: 2 }}>{message}</Typography>
+        <ErrorOutlineIcon style={{ color: "red" }} />
+      )}
+      <Typography style={{ marginLeft: 15, flex: 2 }}>{message}</Typography>
       <button onClick={close} style={buttonStyle}>
         <CloseIcon />
       </button>
     </div>
-  )
+  );
-}
+};
 
-export default AlertTemplate
+export default AlertTemplate;

frontend/src/components/ConfigureWorkflow.jsx

@@ -0,0 +1,677 @@
+import React, { useState } from "react";
+
+import {
+  InputAdornment,
+  Tooltip,
+  TextField,
+  CircularProgress,
+  ButtonGroup,
+  Button,
+  Avatar,
+  ListItemAvatar,
+  Typography,
+  List,
+  ListItem,
+  ListItemText,
+} from "@material-ui/core";
+import { FavoriteBorder as FavoriteBorderIcon } from "@material-ui/icons";
+import { FixName } from "../views/Apps.jsx";
+
+// Handles workflow updates on first open to highlight the issues of the workflow
+// Variables
+// Action (exists, missing fields)
+// Action auth
+// Triggers
+//
+// Specifically used for UNSAVED workflows only?
+const ConfigureWorkflow = (props) => {
+  const {
+    globalUrl,
+    theme,
+    workflow,
+    appAuthentication,
+    setSelectedAction,
+    setAuthenticationModalOpen,
+    setSelectedApp,
+    apps,
+    selectedAction,
+    setConfigureWorkflowModalOpen,
+    saveWorkflow,
+    newWebhook,
+    submitSchedule,
+    referenceUrl,
+    isCloud,
+    setAuthenticationType,
+    alert,
+  } = props;
+  const [requiredActions, setRequiredActions] = React.useState([]);
+  const [requiredVariables, setRequiredVariables] = React.useState([]);
+  const [requiredTriggers, setRequiredTriggers] = React.useState([]);
+  const [previousAuth, setPreviousAuth] = React.useState(appAuthentication);
+  const [firstLoad, setFirstLoad] = React.useState("");
+  const [itemChanged, setItemChanged] = React.useState(false);
+  var finished = false;
+
+  if (workflow === undefined || workflow === null) {
+    return null;
+  }
+
+  if (apps === undefined || apps === null) {
+    return null;
+  }
+
+  if (appAuthentication === undefined || appAuthentication === null) {
+    return null;
+  }
+
+  const getApp = (actionId, appId) => {
+    fetch(globalUrl + "/api/v1/apps/" + appId + "/config?openapi=false", {
+      headers: {
+        Accept: "application/json",
+      },
+      credentials: "include",
+    })
+      .then((response) => {
+        if (response.status === 200) {
+          //alert.success("Successfully GOT app "+appId)
+        } else {
+          alert.error("Failed getting app");
+        }
+
+        return response.json();
+      })
+      .then((responseJson) => {
+        console.log("ACTION: ", responseJson);
+        if (
+          responseJson.actions !== undefined &&
+          responseJson.actions !== null
+        ) {
+        }
+      })
+      .catch((error) => {
+        alert.error(error.toString());
+      });
+  };
+
+  if (firstLoad.length === 0 || firstLoad !== workflow.id) {
+    if (finished) {
+      setConfigureWorkflowModalOpen(false);
+      return null;
+    }
+
+    if (apps === undefined || apps === null || apps.length === 0) {
+      console.log("No apps loaded: ", apps);
+      setConfigureWorkflowModalOpen(false);
+      return null;
+    }
+
+    setFirstLoad(workflow.id);
+    const newactions = [];
+    for (var key in workflow.actions) {
+      const action = workflow.actions[key];
+      var newaction = {
+        large_image: action.large_image,
+        app_name: action.app_name,
+        app_version: action.app_version,
+        activation_done: false,
+        must_activate: false,
+        must_authenticate: false,
+        auth_done: false,
+        action_ids: [],
+        action: action,
+				update_version: action.app_version,
+        app: {},
+      };
+
+      const app = apps.find(
+        (app) =>
+          app.name === action.app_name &&
+          (app.app_version === action.app_version ||
+            (app.loop_versions !== null &&
+              app.loop_versions.includes(action.app_version)))
+      );
+      if (app === undefined || app === null) {
+        //console.log("App not found: ", action.app_name);
+          
+      	const subapp = apps.find(app => app.name === action.app_name)
+				if (subapp !== undefined && subapp !== null) {
+					newaction.update_version = "1.1.0"
+				}
+
+
+        newaction.must_activate = true;
+      } else {
+        if (
+          action.authentication_id === "" &&
+          app.authentication.required === true
+        ) {
+          // Check if configuration is filled or not
+          var filled = true;
+          for (var key in action.parameters) {
+            if (action.parameters[key].configuration) {
+              //console.log("Found config: ", action.parameters[key])
+              if (
+                action.parameters[key].value === null ||
+                action.parameters[key].value.length === 0
+              ) {
+                filled = false;
+                break;
+              }
+            }
+          }
+
+          if (!filled) {
+            newaction.must_authenticate = true;
+            newaction.action_ids.push(action.id);
+          }
+        } else if (action.authentication_id !== "" && app.authentication.required === true) {
+					console.log("Should verify authentication ID ", action.authentication_id)
+
+				}
+
+        newaction.app = app;
+      }
+
+      if (
+        action.errors !== undefined &&
+        action.errors !== null &&
+        action.errors.length > 0
+      ) {
+        //console.log("Node has errors!: ", action.errors)
+      }
+
+      if (newaction.must_authenticate) {
+        var authenticationOptions = [];
+        for (var key in appAuthentication) {
+          const auth = appAuthentication[key];
+          if (auth.app.name === app.name && auth.active) {
+            //console.log("Found auth: ", auth)
+            authenticationOptions.push(auth);
+            newaction.authenticationId = auth.id;
+            break;
+          }
+        }
+
+        if (
+          newaction.authenticationId === null ||
+          newaction.authenticationId === undefined ||
+          newaction.authenticationId.length === ""
+        ) {
+          //console.log("FAILED to authenticate node!")
+
+          if (
+            newactions.find(
+              (tmpaction) =>
+                tmpaction.app_id === newaction.app_id &&
+                tmpaction.app_name === newaction.app_name
+            ) !== undefined
+          ) {
+            //console.log("Action already found.");
+          } else {
+            newactions.push(newaction);
+          }
+        } else {
+          //console.log("Skipping node as it's already authenticated.")
+          newaction.authentication = authenticationOptions;
+          workflow.actions[key] = newaction;
+        }
+      } else if (newaction.must_activate) {
+        if (
+          newactions.find(
+            (tmpaction) =>
+              tmpaction.app_id === newaction.app_id &&
+              tmpaction.app_name === newaction.app_name
+          ) !== undefined
+        ) {
+          console.log("Action already found.");
+        } else {
+          newactions.push(newaction);
+        }
+      }
+    }
+
+    for (var key in workflow.workflow_variables) {
+      const variable = workflow.workflow_variables[key];
+      if (
+        variable.value === undefined ||
+        variable.value === undefined ||
+        variable.value.length < 2
+      ) {
+        variable.value = "";
+        variable.index = key;
+        requiredVariables.push(variable);
+      }
+    }
+
+    for (var key in workflow.triggers) {
+      var trigger = workflow.triggers[key];
+      trigger.index = key;
+
+      if (trigger.status === "running") {
+        continue;
+      }
+
+      if (
+        trigger.trigger_type === "SUBFLOW" ||
+        trigger.trigger_type === "USERINPUT"
+      ) {
+        continue;
+      }
+
+      requiredTriggers.push(trigger);
+    }
+
+    if (
+      requiredTriggers.length === 0 &&
+      requiredVariables.length === 0 &&
+      newactions.length === 0
+    ) {
+      setConfigureWorkflowModalOpen(false);
+    }
+
+    setRequiredTriggers(requiredTriggers);
+    setRequiredVariables(requiredVariables);
+    setRequiredActions(newactions);
+  }
+
+  if (appAuthentication.length !== previousAuth.length) {
+    var newactions = [];
+    for (var actionkey in requiredActions) {
+      var newaction = requiredActions[actionkey];
+      const app = newaction.app;
+
+      for (var key in appAuthentication) {
+        const auth = appAuthentication[key];
+        if (auth.app.name === app.name && auth.active) {
+          newaction.auth_done = true;
+          break;
+        }
+      }
+
+      newactions.push(newaction);
+    }
+
+    setRequiredActions(newactions);
+    setPreviousAuth(appAuthentication);
+    // Set auth done to true
+    //"auth_done": false
+  }
+
+  const TriggerSection = (props) => {
+    const { trigger } = props;
+
+    return (
+      <ListItem>
+        <ListItemAvatar>
+          <Avatar variant="rounded">
+            <img
+              alt={trigger.label}
+              src={trigger.large_image}
+              style={{ width: 50 }}
+            />
+          </Avatar>
+        </ListItemAvatar>
+        <ListItemText
+          primary={trigger.name}
+          secondary={trigger.description}
+          style={{}}
+        />
+        {trigger.trigger_type === "WEBHOOK" && trigger.status !== "running" ? (
+          <Button
+            disabled={trigger.status === "running"}
+            color="primary"
+            variant="contained"
+            onClick={() => {
+              workflow.triggers[trigger.index].status = "running";
+              if (workflow.triggers[trigger.index].parameters === null) {
+                workflow.triggers[trigger.index].parameters = [
+                  {
+                    name: "url",
+                    value: referenceUrl + "webhook_" + trigger.id,
+                  },
+                  { name: "tmp", value: "webhook_" + trigger.id },
+                ];
+              }
+
+              newWebhook(workflow.triggers[trigger.index]);
+              saveWorkflow(workflow);
+              setItemChanged(true);
+            }}
+          >
+            {trigger.status !== "running" ? "Start" : "Running"}
+          </Button>
+        ) : trigger.trigger_type === "SCHEDULE" &&
+          trigger.status !== "running" ? (
+          <Button
+            disabled={trigger.status === "running"}
+            color="primary"
+            variant="contained"
+            onClick={() => {
+              workflow.triggers[trigger.index].status = "running";
+              if (workflow.triggers[trigger.index].parameters === null) {
+                workflow.triggers[trigger.index].parameters = [
+                  { name: "cron", value: isCloud ? "*/15 * * * *" : "120" },
+                  {
+                    name: "execution_argument",
+                    value: '{"example": {"json": "is cool"}}',
+                  },
+                ];
+              }
+
+              submitSchedule(workflow.triggers[trigger.index], trigger.index);
+              saveWorkflow(workflow);
+              setItemChanged(true);
+            }}
+          >
+            {trigger.status !== "running" ? "Start" : "Running"}
+          </Button>
+        ) : null}
+        {/*
+					<ListItemText
+						primary={
+							<TextField
+								style={{backgroundColor: theme.palette.inputColor, borderRadius: 5,}} 
+								InputProps={{
+									style:{
+										color: "white",
+										minHeight: 50, 
+										marginLeft: 5,
+										maxWidth: "95%",
+										fontSize: "1em",
+									},
+									endAdornment: (
+										<InputAdornment position="end">
+										</InputAdornment>
+									)
+								}}
+								fullWidth
+								color="primary"
+								type={"text"}
+								placeholder={`New value for ${trigger.name}`}
+								onChange={(event) => {
+									console.log("NEW VALUE ON INDEX", trigger.value)
+								}}
+								onBlur={(event) => {
+									//workflow.variables[variable.index] = event.target.value
+								}}
+							/>
+							}
+							style={{}}
+						/>
+						*/}
+      </ListItem>
+    );
+  };
+
+  const VariableSection = (props) => {
+    const { variable } = props;
+
+    //<Typography variant="body2">Name: {variable.name} - {variable.value}. </Typography>
+    return (
+      <ListItem>
+        <ListItemAvatar>
+          <Avatar>
+            <FavoriteBorderIcon />
+          </Avatar>
+        </ListItemAvatar>
+        <ListItemText
+          primary={variable.name}
+          secondary={variable.description}
+          style={{}}
+        />
+        <ListItemText
+          primary={
+            <TextField
+              style={{
+                backgroundColor: theme.palette.inputColor,
+                borderRadius: 5,
+              }}
+              InputProps={{
+                style: {
+                  color: "white",
+                  minHeight: 50,
+                  marginLeft: 5,
+                  maxWidth: "95%",
+                  fontSize: "1em",
+                },
+                endAdornment: <InputAdornment position="end"></InputAdornment>,
+              }}
+              fullWidth
+              color="primary"
+              type={"text"}
+              placeholder={`New value for ${variable.name}`}
+              onChange={(event) => {
+                console.log(
+                  "NEW VALUE ON INDEX",
+                  variable.index,
+                  variable.value
+                );
+              }}
+              onBlur={(event) => {
+                workflow.workflow_variables[variable.index].value =
+                  event.target.value;
+              }}
+            />
+          }
+          style={{}}
+        />
+      </ListItem>
+    );
+  };
+
+  const activateApp = (app_id, app_name, app_version) => {
+    fetch(
+      `${globalUrl}/api/v1/apps/${app_id}/activate?app_name=${app_name}&app_version=${app_version}`,
+      {
+        method: "GET",
+        headers: {
+          "Content-Type": "application/json",
+          Accept: "application/json",
+        },
+        credentials: "include",
+      }
+    )
+      .then((response) => {
+        if (response.status !== 200) {
+          //window.location.pathname = "/search"
+          //alert.error("Failed to find this app. Is it public?")
+        }
+
+        return response.json();
+      })
+      .then((responseJson) => {
+        if (responseJson.success === false) {
+        	if (responseJson.reason !== undefined) {
+          	alert.error("Failed to activate the app: "+responseJson.reason);
+					} else {
+          	alert.error("Failed to activate the app");
+					}
+        } else {
+          alert.success("App activated for your organization!");
+        }
+      })
+      .catch((error) => {
+        alert.error(error.toString());
+      });
+  };
+
+
+
+  const AppSection = (props) => {
+    const { action } = props;
+
+    return (
+      <ListItem>
+        <ListItemAvatar>
+          <Avatar variant="rounded">
+            <img
+              alt={action.app_name}
+              src={action.large_image}
+              style={{ width: 50 }}
+            />
+          </Avatar>
+        </ListItemAvatar>
+        <ListItemText
+          primary={FixName(action.app_name)}
+          secondary={action.app_version}
+          style={{}}
+        />
+        {action.must_authenticate ? (
+          action.auth_done ? (
+            <Button color="primary" variant="outlined" onClick={() => {}}>
+              Authenticated
+            </Button>
+          ) : selectedAction.app_name === action.app_name ? (
+            <CircularProgress />
+          ) : (
+            <Button
+              color="primary"
+              variant="contained"
+              onClick={() => {
+                setAuthenticationType(
+                  action.app.authentication.type === "oauth2" &&
+                    action.app.authentication.redirect_uri !== undefined &&
+                    action.app.authentication.redirect_uri !== null
+                    ? {
+                        type: "oauth2",
+                        redirect_uri: action.app.authentication.redirect_uri,
+                        token_uri: action.app.authentication.token_uri,
+                        scope: action.app.authentication.scope,
+                      }
+                    : {
+                        type: "",
+                      }
+                );
+
+                setItemChanged(true);
+                setSelectedAction(action.action);
+                setSelectedApp(action.app);
+                setAuthenticationModalOpen(true);
+              }}
+            >
+              Authenticate
+            </Button>
+          )
+        ) : null}
+        {action.must_activate ? (
+          <Button
+            color="primary"
+            variant="contained"
+            onClick={() => {
+							console.log("ACTION: ", action)
+              activateApp(action.action.app_id, action.app_name, action.app_version);
+              setItemChanged(true);
+            }}
+          >
+            Activate
+          </Button>
+        ) : null}
+				{action.update_version !== action.app_version ? (
+          <Button
+            color="primary"
+            variant="contained"
+						style={{marginLeft: 5}}
+            onClick={() => {
+							console.log("Set version to: ", action.update_version)
+
+							if (workflow.actions !== null) {
+								//console.log(workflow.actions)
+								alert.info("Setting action to version "+action.update_version)
+								for (var key in workflow.actions) {
+									if (workflow.actions[key].app_name === action.app_name && workflow.actions[key].app_version === action.app_version) {
+										workflow.actions[key].app_version = action.update_version
+
+										if (!itemChanged) {
+											setItemChanged(true)
+										}
+									}
+								}
+
+								action.must_activate = false
+								action.update_version = action.app_version
+							}
+            }}
+          >
+            {action.update_version}
+          </Button>
+        ) : null}
+      </ListItem>
+    );
+  };
+
+  return (
+    <div>
+      <Typography variant="h6">{workflow.name}</Typography>
+      <Typography variant="body1" color="textSecondary">
+        The following configuration makes the workflow ready immediately.
+      </Typography>
+      {requiredActions.length > 0 ? (
+        <span>
+          <Typography variant="body1" style={{ marginTop: 10 }}>
+            Actions
+          </Typography>
+          <List>
+            {requiredActions.map((data, index) => {
+              return <AppSection key={index} action={data} />;
+            })}
+          </List>
+        </span>
+      ) : null}
+
+      {requiredVariables.length > 0 ? (
+        <span>
+          <Typography variant="body1" style={{ marginTop: 10 }}>
+            Variables
+          </Typography>
+          <List>
+            {requiredVariables.map((data, index) => {
+              return <VariableSection key={index} variable={data} />;
+            })}
+          </List>
+        </span>
+      ) : null}
+
+      {requiredTriggers.length > 0 ? (
+        <span>
+          <Typography variant="body1" style={{ marginTop: 10 }}>
+            Triggers
+          </Typography>
+          <List>
+            {requiredTriggers.map((data, index) => {
+              return <TriggerSection key={index} trigger={data} />;
+            })}
+          </List>
+        </span>
+      ) : null}
+      <div style={{ textAlign: "center", display: "flex", marginTop: 20 }}>
+        <ButtonGroup style={{ margin: "auto" }}>
+          {/*
+					<Button color="primary" variant={"outlined"} style={{
+					}} onClick={() => {
+						setConfigureWorkflowModalOpen(false)
+					}}>
+						Skip 
+					</Button>
+					*/}
+          <Button
+            color="primary"
+            variant={itemChanged ? "contained" : "outlined"}
+            style={{}}
+            onClick={() => {
+              if (itemChanged) {
+                saveWorkflow(workflow);
+                window.location.reload();
+              } else {
+                setConfigureWorkflowModalOpen(false);
+              }
+            }}
+          >
+          	Close window 
+          </Button>
+        </ButtonGroup>
+      </div>
+    </div>
+  );
+};
+
+export default ConfigureWorkflow;

frontend/src/components/Dropzone.js

@@ -1,19 +1,19 @@
-import React, { useRef, useState } from 'react';
+import React, { useRef, useState } from "react";
-import { useEffect } from 'react';
+import { useEffect } from "react";
-import BackupIcon from '@material-ui/icons/Backup';
+import BackupIcon from "@material-ui/icons/Backup";
 
 const dragOverStyle = {
-  backgroundColor: 'rgba(0,0,0,0.8)',
+  backgroundColor: "rgba(0,0,0,0.8)",
-  border: '5px dashed white',
+  border: "5px dashed white",
-  borderRadius: '8px',
+  borderRadius: "8px",
-  width: '100%',
+  width: "100%",
-  height: '100%',
+  height: "100%",
-  position: 'absolute',
+  position: "absolute",
-  overflow: 'hidden',
+  overflow: "hidden",
   zIndex: 100,
-  display: 'flex',
+  display: "flex",
-  alignItems: 'center',
+  alignItems: "center",
-  justifyContent: 'center'
+  justifyContent: "center",
 };
 
 const Dropzone = ({ children, style, onDrop }) => {
@@ -56,21 +56,21 @@ const Dropzone = ({ children, style, onDrop }) => {
   useEffect(() => {
     if (!dropzoneRef.current) return;
 
-    dropzoneRef.current.addEventListener('dragover', handleDragOver);
+    dropzoneRef.current.addEventListener("dragover", handleDragOver);
-    dropzoneRef.current.addEventListener('dragenter', handleDragEnter);
+    dropzoneRef.current.addEventListener("dragenter", handleDragEnter);
-    dropzoneRef.current.addEventListener('dragleave', handleDragLeave);
+    dropzoneRef.current.addEventListener("dragleave", handleDragLeave);
-    dropzoneRef.current.addEventListener('drop', handleDrop);
+    dropzoneRef.current.addEventListener("drop", handleDrop);
 
     return () => {
-      dropzoneRef.current.removeEventListener('dragover', handleDragOver);
+      dropzoneRef.current.removeEventListener("dragover", handleDragOver);
-      dropzoneRef.current.removeEventListener('dragenter', handleDragEnter);
+      dropzoneRef.current.removeEventListener("dragenter", handleDragEnter);
-      dropzoneRef.current.removeEventListener('dragleave', handleDragLeave);
+      dropzoneRef.current.removeEventListener("dragleave", handleDragLeave);
-      dropzoneRef.current.removeEventListener('drop', handleDrop);
+      dropzoneRef.current.removeEventListener("drop", handleDrop);
     };
   }, [dropzoneRef]);
 
   return (
-    <div ref={dropzoneRef} style={{ position: 'relative', ...style }}>
+    <div ref={dropzoneRef} style={{ position: "relative", ...style }}>
       {dragging && (
         <div style={dragOverStyle}>
           <BackupIcon fontSize="large" />

frontend/src/components/FAQ.jsx

@@ -0,0 +1,23 @@
+import React, {useState} from 'react';
+
+
+const FAQItem = (props) => {
+	const { question, answer } = props
+
+	const [isExpanded, setIsExpanded] = useState(false)
+
+	return (
+		<Paper onClick={() => {
+			setIsExpanded(!isExpanded)	
+		}}>
+			<Typography variant="body1">
+				{question}
+			</Typography>
+			<Typography variant="body2" color="textSecondary">
+				{answer}
+			</Typography>
+		</Paper>
+	)
+}
+
+export default FAQItem;

frontend/src/components/FooterNew.js

@@ -1,54 +1,54 @@
-import React from 'react';
+import React from "react";
 
 //import List from '@material-ui/core/List';
 //import ListItem from '@material-ui/core/ListItem';
 
 //borderTop: "1px solid #385F71"
 const FooterStyle = {
-	right: "0",
+  right: "0",
-	left: "0",
+  left: "0",
-	bottom: "0",
+  bottom: "0",
-	height: "130px",
+  height: "130px",
-	backgroundColor: 'rgba(15, 14, 31, 1)',
+  backgroundColor: "rgba(15, 14, 31, 1)",
 };
 
 const FooterInfo = {
-  maxWidth: '1150px',
+  maxWidth: "1150px",
-  minWidth: '768px',
+  minWidth: "768px",
-  textAlign: 'center',
+  textAlign: "center",
-  margin: 'auto',
+  margin: "auto",
 };
 
 const hrefStyle = {
-	color: "#bdbdbd", 
+  color: "#bdbdbd",
-	textDecoration: "none"
+  textDecoration: "none",
-}
-
-const Footer = props => {
-  	return (
-		<div style={FooterStyle}>
-			<div style={FooterInfo}>
-				<Box />
-			</div>
-		</div>
-  	);
 };
 
-const Box = props => {
+const Footer = (props) => {
-  return(
+  return (
-		<div style={{display: "flex"}}>
+    <div style={FooterStyle}>
-			<div style={{flex: "1"}}>
+      <div style={FooterInfo}>
-				<a style={hrefStyle} href="/about">
+        <Box />
-					<h1>About</h1>
+      </div>
-				</a>
+    </div>
-			</div>
+  );
-			<div style={{flex: "1"}}>
+};
-				<a style={hrefStyle} href="/privacy-policy">
+
-					<h1>Privacy Policy</h1>
+const Box = (props) => {
-				</a>
+  return (
-			</div>
+    <div style={{ display: "flex" }}>
-		</div>
+      <div style={{ flex: "1" }}>
-	);
+        <a style={hrefStyle} href="/about">
+          <h1>About</h1>
+        </a>
+      </div>
+      <div style={{ flex: "1" }}>
+        <a style={hrefStyle} href="/privacy-policy">
+          <h1>Privacy Policy</h1>
+        </a>
+      </div>
+    </div>
+  );
 };
 
 export default Footer;

frontend/src/components/LandingpageUsecases.jsx

@@ -0,0 +1,245 @@
+import React, { useState, useEffect } from 'react';
+import {isMobile} from "react-device-detect";
+import DetectionFramework, { usecases } from "../components/DetectionFramework.jsx";
+import {Link} from 'react-router-dom';
+import ReactGA from 'react-ga';
+
+import { Button, LinearProgress, Typography } from '@material-ui/core';
+
+export const securityFramework = [
+		{
+			image: <path d="M15.6408 8.39233H18.0922V10.0287H15.6408V8.39233ZM0.115234 8.39233H2.56663V10.0287H0.115234V8.39233ZM9.92083 0.21051V2.66506H8.28656V0.21051H9.92083ZM3.31839 2.25596L5.05889 4.00687L3.89856 5.16051L2.15807 3.42596L3.31839 2.25596ZM13.1485 3.99869L14.8808 2.25596L16.0493 3.42596L14.3088 5.16051L13.1485 3.99869ZM9.10369 4.30142C10.404 4.30142 11.651 4.81863 12.5705 5.73926C13.4899 6.65989 14.0065 7.90854 14.0065 9.21051C14.0065 11.0269 13.0178 12.6141 11.5551 13.4651V14.9378C11.5551 15.1548 11.469 15.3629 11.3158 15.5163C11.1625 15.6698 10.9547 15.756 10.738 15.756H7.46943C7.25271 15.756 7.04487 15.6698 6.89163 15.5163C6.73839 15.3629 6.6523 15.1548 6.6523 14.9378V13.4651C5.18963 12.6141 4.2009 11.0269 4.2009 9.21051C4.2009 7.90854 4.71744 6.65989 5.63689 5.73926C6.55635 4.81863 7.80339 4.30142 9.10369 4.30142ZM10.738 16.5741V17.3923C10.738 17.6093 10.6519 17.8174 10.4986 17.9709C10.3454 18.1243 10.1375 18.2105 9.92083 18.2105H8.28656C8.06984 18.2105 7.862 18.1243 7.70876 17.9709C7.55552 17.8174 7.46943 17.6093 7.46943 17.3923V16.5741H10.738ZM8.28656 14.1196H9.92083V12.3769C11.3345 12.0169 12.3722 10.7323 12.3722 9.21051C12.3722 8.34253 12.0279 7.5101 11.4149 6.89634C10.8019 6.28259 9.97056 5.93778 9.10369 5.93778C8.23683 5.93778 7.40546 6.28259 6.79249 6.89634C6.17953 7.5101 5.83516 8.34253 5.83516 9.21051C5.83516 10.7323 6.87292 12.0169 8.28656 12.3769V14.1196Z" />,
+			text: "Cases",
+			description: "Case management"
+		}, 
+		{
+			image: 
+			<path d="M6.93767 0C8.71083 0 10.4114 0.704386 11.6652 1.9582C12.919 3.21202 13.6234 4.91255 13.6234 6.68571C13.6234 8.34171 13.0165 9.864 12.0188 11.0366L12.2965 11.3143H13.1091L18.252 16.4571L16.7091 18L11.5662 12.8571V12.0446L11.2885 11.7669C10.116 12.7646 8.59367 13.3714 6.93767 13.3714C5.16451 13.3714 3.46397 12.667 2.21015 11.4132C0.956339 10.1594 0.251953 8.45888 0.251953 6.68571C0.251953 4.91255 0.956339 3.21202 2.21015 1.9582C3.46397 0.704386 5.16451 0 6.93767 0ZM6.93767 2.05714C4.36624 2.05714 2.3091 4.11429 2.3091 6.68571C2.3091 9.25714 4.36624 11.3143 6.93767 11.3143C9.5091 11.3143 11.5662 9.25714 11.5662 6.68571C11.5662 4.11429 9.5091 2.05714 6.93767 2.05714Z" />,
+			text: "SIEM",
+			description: "Case management"
+		}, 
+		{
+			image: 
+			<path d="M11.223 10.971L3.85195 14.4L7.28095 7.029L14.652 3.6L11.223 10.971ZM9.25195 0C8.07006 0 6.89973 0.232792 5.8078 0.685084C4.71587 1.13738 3.72372 1.80031 2.88799 2.63604C1.20016 4.32387 0.251953 6.61305 0.251953 9C0.251953 11.3869 1.20016 13.6761 2.88799 15.364C3.72372 16.1997 4.71587 16.8626 5.8078 17.3149C6.89973 17.7672 8.07006 18 9.25195 18C11.6389 18 13.9281 17.0518 15.6159 15.364C17.3037 13.6761 18.252 11.3869 18.252 9C18.252 7.8181 18.0192 6.64778 17.5669 5.55585C17.1146 4.46392 16.4516 3.47177 15.6159 2.63604C14.7802 1.80031 13.788 1.13738 12.6961 0.685084C11.6042 0.232792 10.4338 0 9.25195 0ZM9.25195 8.01C8.98939 8.01 8.73758 8.1143 8.55192 8.29996C8.36626 8.48563 8.26195 8.73744 8.26195 9C8.26195 9.26256 8.36626 9.51437 8.55192 9.70004C8.73758 9.8857 8.98939 9.99 9.25195 9.99C9.51452 9.99 9.76633 9.8857 9.95199 9.70004C10.1376 9.51437 10.242 9.26256 10.242 9C10.242 8.73744 10.1376 8.48563 9.95199 8.29996C9.76633 8.1143 9.51452 8.01 9.25195 8.01Z" />,
+			text: "Assets",
+			description: "Case management"
+		}, 
+		{
+			image: 
+			<path d="M13.3318 2.223C13.2598 2.223 13.1878 2.205 13.1248 2.169C11.3968 1.278 9.90284 0.9 8.11184 0.9C6.32984 0.9 4.63784 1.323 3.09884 2.169C2.88284 2.286 2.61284 2.205 2.48684 1.989C2.36984 1.773 2.45084 1.494 2.66684 1.377C4.34084 0.468 6.17684 0 8.11184 0C10.0288 0 11.7028 0.423 13.5388 1.368C13.7638 1.485 13.8448 1.755 13.7278 1.971C13.6468 2.133 13.4938 2.223 13.3318 2.223ZM0.452843 6.948C0.362843 6.948 0.272843 6.921 0.191843 6.867C-0.015157 6.723 -0.0601571 6.444 0.0838429 6.237C0.974843 4.977 2.10884 3.987 3.45884 3.294C6.28484 1.836 9.90284 1.827 12.7378 3.285C14.0878 3.978 15.2218 4.959 16.1128 6.21C16.2568 6.408 16.2118 6.696 16.0048 6.84C15.7978 6.984 15.5188 6.939 15.3748 6.732C14.5648 5.598 13.5388 4.707 12.3238 4.086C9.74084 2.763 6.43784 2.763 3.86384 4.095C2.63984 4.725 1.61384 5.625 0.803843 6.759C0.731843 6.885 0.596843 6.948 0.452843 6.948ZM6.07784 17.811C5.96084 17.811 5.84384 17.766 5.76284 17.676C4.97984 16.893 4.55684 16.389 3.95384 15.3C3.33284 14.193 3.00884 12.843 3.00884 11.394C3.00884 8.721 5.29484 6.543 8.10284 6.543C10.9108 6.543 13.1968 8.721 13.1968 11.394C13.1968 11.646 12.9988 11.844 12.7468 11.844C12.4948 11.844 12.2968 11.646 12.2968 11.394C12.2968 9.216 10.4158 7.443 8.10284 7.443C5.78984 7.443 3.90884 9.216 3.90884 11.394C3.90884 12.69 4.19684 13.887 4.74584 14.859C5.32184 15.894 5.71784 16.335 6.41084 17.037C6.58184 17.217 6.58184 17.496 6.41084 17.676C6.31184 17.766 6.19484 17.811 6.07784 17.811ZM12.5308 16.146C11.4598 16.146 10.5148 15.876 9.74084 15.345C8.39984 14.436 7.59884 12.96 7.59884 11.394C7.59884 11.142 7.79684 10.944 8.04884 10.944C8.30084 10.944 8.49884 11.142 8.49884 11.394C8.49884 12.663 9.14684 13.86 10.2448 14.598C10.8838 15.03 11.6308 15.237 12.5308 15.237C12.7468 15.237 13.1068 15.21 13.4668 15.147C13.7098 15.102 13.9438 15.264 13.9888 15.516C14.0338 15.759 13.8718 15.993 13.6198 16.038C13.1068 16.137 12.6568 16.146 12.5308 16.146ZM10.7218 18C10.6858 18 10.6408 17.991 10.6048 17.982C9.17384 17.586 8.23784 17.055 7.25684 16.092C5.99684 14.841 5.30384 13.176 5.30384 11.394C5.30384 9.936 6.54584 8.748 8.07584 8.748C9.60584 8.748 10.8478 9.936 10.8478 11.394C10.8478 12.357 11.6848 13.14 12.7198 13.14C13.7548 13.14 14.5918 12.357 14.5918 11.394C14.5918 8.001 11.6668 5.247 8.06684 5.247C5.51084 5.247 3.17084 6.669 2.11784 8.874C1.76684 9.603 1.58684 10.458 1.58684 11.394C1.58684 12.096 1.64984 13.203 2.18984 14.643C2.27984 14.877 2.16284 15.138 1.92884 15.219C1.69484 15.309 1.43384 15.183 1.35284 14.958C0.911843 13.779 0.695843 12.609 0.695843 11.394C0.695843 10.314 0.902843 9.333 1.30784 8.478C2.50484 5.967 5.15984 4.338 8.06684 4.338C12.1618 4.338 15.4918 7.497 15.4918 11.385C15.4918 12.843 14.2498 14.031 12.7198 14.031C11.1898 14.031 9.94784 12.843 9.94784 11.385C9.94784 10.422 9.11084 9.639 8.07584 9.639C7.04084 9.639 6.20384 10.422 6.20384 11.385C6.20384 12.924 6.79784 14.364 7.88684 15.444C8.74184 16.29 9.56084 16.758 10.8298 17.109C11.0728 17.172 11.2078 17.424 11.1448 17.658C11.0998 17.865 10.9108 18 10.7218 18Z" />,
+			text: "IAM",
+			description: "Case management"
+		}, 
+		{
+			image:  <path d="M16.1091 8.57143H14.8234V5.14286C14.8234 4.19143 14.052 3.42857 13.1091 3.42857H9.68052V2.14286C9.68052 1.57454 9.45476 1.02949 9.0529 0.627628C8.65103 0.225765 8.10599 0 7.53767 0C6.96935 0 6.4243 0.225765 6.02244 0.627628C5.62057 1.02949 5.39481 1.57454 5.39481 2.14286V3.42857H1.96624C1.51158 3.42857 1.07555 3.60918 0.754056 3.93067C0.432565 4.25216 0.251953 4.6882 0.251953 5.14286V8.4H1.53767C2.82338 8.4 3.85195 9.42857 3.85195 10.7143C3.85195 12 2.82338 13.0286 1.53767 13.0286H0.251953V16.2857C0.251953 16.7404 0.432565 17.1764 0.754056 17.4979C1.07555 17.8194 1.51158 18 1.96624 18H5.22338V16.7143C5.22338 15.4286 6.25195 14.4 7.53767 14.4C8.82338 14.4 9.85195 15.4286 9.85195 16.7143V18H13.1091C13.5638 18 13.9998 17.8194 14.3213 17.4979C14.6428 17.1764 14.8234 16.7404 14.8234 16.2857V12.8571H16.1091C16.6774 12.8571 17.2225 12.6314 17.6243 12.2295C18.0262 11.8277 18.252 11.2826 18.252 10.7143C18.252 10.146 18.0262 9.60092 17.6243 9.19906C17.2225 8.79719 16.6774 8.57143 16.1091 8.57143Z" />,
+			text: "Intel",
+			description: "Case management"
+		}, 
+		{
+			image: 
+			<path d="M9.89516 7.71433H8.60945V5.1429H9.89516V7.71433ZM9.89516 10.2858H8.60945V9.00004H9.89516V10.2858ZM14.3952 2.57147H4.10944C3.76845 2.57147 3.44143 2.70693 3.20031 2.94805C2.95919 3.18917 2.82373 3.51619 2.82373 3.85719V15.4286L5.39516 12.8572H14.3952C14.7362 12.8572 15.0632 12.7217 15.3043 12.4806C15.5454 12.2395 15.6809 11.9125 15.6809 11.5715V3.85719C15.6809 3.14361 15.1023 2.57147 14.3952 2.57147Z" />,
+			text: "Comms",
+			description: "Case management"
+		}, 
+		{
+			image: 
+			<path d="M0.251953 10.6011H3.8391L9.38052 -4.92572e-08L10.8977 11.5696L15.0377 6.28838L19.3191 10.6011H23.3948V13.1836H18.252L15.2562 10.175L9.1491 18L7.88909 8.41894L5.39481 13.1836H0.251953V10.6011Z" />,
+			text: "Network",
+			description: "Case management"
+		}, 
+		{
+			image:
+			<path d="M19.1722 8.9957L17.0737 6.60487L17.3661 3.44004L14.2615 2.73483L12.6361 -3.28068e-08L9.71206 1.25561L6.78803 -3.28068e-08L5.16261 2.73483L2.05797 3.43144L2.35038 6.59627L0.251953 8.9957L2.35038 11.3865L2.05797 14.56L5.16261 15.2652L6.78803 18L9.71206 16.7358L12.6361 17.9914L14.2615 15.2566L17.3661 14.5514L17.0737 11.3865L19.1722 8.9957ZM10.5721 13.2957H8.85205V11.5757H10.5721V13.2957ZM10.5721 9.85571H8.85205V4.69565H10.5721V9.85571Z" />,
+			text: "EDR & AV",
+			description: "Case management"
+		}, 
+]
+
+const LandingpageUsecases = (props) => {
+	const [selectedUsecase, setSelectedUsecase] = useState("Phishing")
+	const usecasekeys = usecases === undefined || usecases === null ? [] : Object.keys(usecases)
+	const buttonBackground = "linear-gradient(to right, #f86a3e, #f34079)"
+	const buttonStyle = {borderRadius: 25, height: 50, width: 260, margin: isMobile ? "15px auto 15px auto" : 20, fontSize: 18, backgroundImage: buttonBackground}
+
+	const HandleTitle = (props) => {
+		const { usecases, selectedUsecase, setSelecedUsecase } = props
+		const [progress, setProgress] = useState(0)
+
+  	useEffect(() => {
+			const timer = setInterval(() => {
+				setProgress((oldProgress) => {
+					if (oldProgress >= 105) {
+						const foundIndex = usecasekeys.findIndex(key => key === selectedUsecase)
+						var newitem = usecasekeys[foundIndex+1]
+						if (newitem === undefined || newitem === 0) {
+							newitem = usecasekeys[1]
+						}
+
+						setSelectedUsecase(newitem)
+						return -18
+					}
+
+					if (oldProgress >= 65) {
+						return oldProgress + 3
+					}
+
+					if (oldProgress >= 80) {
+						return oldProgress + 1
+					}
+
+					return oldProgress + 6
+				})
+			}, 165)
+
+			return () => {
+				clearInterval(timer)
+			}
+		}, [])
+
+		if (usecases === null || usecases === undefined || usecases.length === 0) {
+			return null
+		}
+
+		const modifier = isMobile ? 17 : 22
+		return (
+			<span style={{margin: "auto", textAlign: isMobile ? "center" : "left", width: isMobile ? 280 : "100%",}}>
+				<b>Handle <br/>
+				<span style={{marginBottom: 10}}>
+					<i id="usecase-text">{selectedUsecase}</i>
+					<LinearProgress variant="determinate" value={progress} style={{marginTop: 0, marginBottom: 0, height: 3, width: isMobile ? "100%" : selectedUsecase.length*modifier, borderRadius: 10, }} />
+				</span>
+				with confidence</b>
+			</span>
+		)
+	}
+
+	const parsedWidth = isMobile ? "100%" : 1100 
+	return (
+		<div style={{width: isMobile ? null : parsedWidth, margin: isMobile ? "0px 0px 0px 0px" : "auto", color: "white", textAlign: isMobile ? "center" : "left",}}>
+			<div style={{display: "flex", position: "relative",}}>
+				<div style={{maxWidth: isMobile ? "100%" : 420, paddingTop: isMobile ? 0 : 120, zIndex: 1000, margin: "auto",}}>
+
+					<Typography variant="h1" style={{margin: "auto", width: isMobile ? 280 : "100%", marginTop: isMobile ? 50 : 0}}>
+						<HandleTitle usecases={usecases} selectedUsecase={selectedUsecase} setSelectedUsecase={setSelectedUsecase} />
+
+						{/*<b>Security Automation <i>is Hard</i></b>*/}
+					</Typography>
+					<Typography variant="h6" style={{marginTop: isMobile ? 15 : 0,}}>
+						Connecting your everchanging environment is hard. We get it! That's why we built Shuffle, where you can use and share your security workflows to everyones benefit.
+						{/*Shuffle is an automation platform where you don't need to be an expert to automate. Get access to our large pool of security playbooks, apps and people.*/}
+					</Typography>
+					<div style={{display: "flex", textAlign: "center", itemAlign: "center",}}>
+						{isMobile ? null :
+							<Link rel="noopener noreferrer" to={"/pricing"} style={{textDecoration: "none"}}>
+								<Button
+									variant="contained"
+									onClick={() => {
+										ReactGA.event({
+											category: "landingpage",
+											action: "click_main_pricing",
+											label: "",
+										})
+									}}
+									style={{
+										borderRadius: 25, height: 40, width: 175, margin: "15px 0px 15px 0px", fontSize: 14, color: "white", backgroundImage: buttonBackground, marginRight: 10, 
+									}}>
+									See Pricing	
+								</Button>
+							</Link>
+						}
+						{isMobile ? null :
+							<Link rel="noopener noreferrer" to={"/register?message=You'll need to sign up first. No name, company or credit card required."} style={{textDecoration: "none"}}>
+								<Button
+									variant="contained"
+									onClick={() => {
+										ReactGA.event({
+											category: "landingpage",
+											action: "click_main_try_it_out",
+											label: "",
+										})
+									}}
+									style={{
+										borderRadius: 25, height: 40, width: 175, margin: "15px 0px 15px 0px", fontSize: 14, color: "white", backgroundImage: buttonBackground,
+									}}>
+									Start for free	
+								</Button>
+							</Link>
+						}
+					</div>
+				</div>
+				{isMobile ? null : 
+					<div style={{marginLeft: 200, marginTop: 125, zIndex: 1000}}>
+						<DetectionFramework showOptions={false} selectedOption={selectedUsecase} rolling={true} />
+					</div>
+				}
+				{isMobile ? null : 
+				<div style={{position: "absolute", top: 50, right: -200, zIndex: 0,  }}>
+					<svg width="351" height="433" viewBox="0 0 351 433" fill="none" xmlns="http://www.w3.org/2000/svg" style={{zIndex: 0, }}>
+					<path d="M167.781 184.839C167.781 235.244 208.625 276.104 259.03 276.104C309.421 276.104 350.28 235.244 350.28 184.839C350.28 134.448 309.421 93.5892 259.03 93.5892C208.625 93.5741 167.781 134.433 167.781 184.839ZM330.387 184.839C330.387 224.263 298.439 256.195 259.03 256.195C219.621 256.195 187.674 224.248 187.674 184.839C187.674 145.43 219.636 113.483 259.03 113.483C298.439 113.483 330.387 145.43 330.387 184.839Z" fill="white" fill-opacity="0.2"/>
+					<path d="M167.781 387.368C167.781 412.578 188.203 433 213.398 433C238.593 433 259.03 412.578 259.03 387.368C259.03 362.157 238.608 341.735 213.398 341.735C188.187 341.735 167.781 362.172 167.781 387.368ZM249.076 387.368C249.076 407.08 233.095 423.046 213.398 423.046C193.686 423.046 177.72 407.065 177.72 387.368C177.72 367.671 193.686 351.69 213.398 351.69C233.095 351.705 249.076 367.671 249.076 387.368Z" fill="white" fill-opacity="0.2"/>
+					<path d="M56.8637 0.738726C25.7052 0.738724 0.44632 25.9976 0.446317 57.1561C0.446314 88.3146 25.7052 113.573 56.8637 113.573C88.0221 113.573 113.281 88.3146 113.281 57.1561C113.281 25.9977 88.0222 0.738729 56.8637 0.738726Z" fill="white" fill-opacity="0.2"/>
+					</svg>
+				</div>
+				}
+			</div>
+			<div style={{display: "flex", width: isMobile ? "100%" : 300, itemAlign: "center", margin: "auto", marginTop: 20, flexDirection: isMobile ? "column" : "row", textAlign: "center",}}>
+				{isMobile ?
+				<Link rel="noopener noreferrer" to={"/pricing"} style={{textDecoration: "none"}}>
+					<Button
+						variant={isMobile ? "contained" : "outlined"}
+						color={isMobile ? "primary" : "secondary"}
+						style={buttonStyle}
+						onClick={() => {
+							ReactGA.event({
+								category: "landingpage",
+								action: "click_main_pricing",
+								label: "",
+							})
+						}}
+						>
+						See pricing 
+					</Button>
+				</Link>
+				: null
+				}
+				{/*isMobile ? 
+				<Link rel="noopener noreferrer" to={"/docs/features"} style={{textDecoration: "none"}}>
+					<Button
+						variant="outlined"
+						onClick={() => {
+							ReactGA.event({
+								category: "landingpage",
+								action: "click_main_features",
+								label: "",
+							})
+						}}
+						color="secondary"
+						style={buttonStyle}>
+						Features 
+					</Button>
+						</Link>
+						: null*/}
+					</div>
+			{isMobile ? null : 
+				<div style={{display: "flex", width: parsedWidth, margin: "auto", marginTop: 150}}>
+					{securityFramework.map((data, index) => {
+						return (
+							<div key={index} style={{flex: 1, textAlign: "center",}}>
+								<span style={{margin: "auto", width: 25,}}>
+									<svg width="25" height="25" fill="white" xmlns="http://www.w3.org/2000/svg" >
+										{data.image}
+									</svg>
+								</span>
+								<Typography variant="body2" style={{color: "white", marginRight: 5}}>
+									{data.text}
+								</Typography>
+							</div>
+						)
+					})}
+				</div>
+			}
+			</div>
+		)
+}
+
+export default LandingpageUsecases;

frontend/src/components/LoginPopup.js

@@ -1,139 +1,176 @@
 /* eslint-disable react/no-multi-comp */
-import React, {useState} from 'react';
+import React, { useState } from "react";
 
-import DialogTitle from '@material-ui/core/DialogTitle';
+import DialogTitle from "@material-ui/core/DialogTitle";
-import Dialog from '@material-ui/core/Dialog';
+import Dialog from "@material-ui/core/Dialog";
-import TextField from '@material-ui/core/TextField';
+import TextField from "@material-ui/core/TextField";
-import Button from '@material-ui/core/Button';
+import Button from "@material-ui/core/Button";
 
-const LoginDialog = props => {
+const LoginDialog = (props) => {
-  	const { classes, onClose, open, globalUrl, isLoggedIn, setIsLoggedIn, ...other } = props;
+  const {
+    classes,
+    onClose,
+    open,
+    globalUrl,
+    isLoggedIn,
+    setIsLoggedIn,
+    ...other
+  } = props;
 
-  	const [username, setUsername] = useState("");
+  const [username, setUsername] = useState("");
-  	const [password, setPassword] = useState("");
+  const [password, setPassword] = useState("");
-  	//const [selectedValue, setSelectedValue] = useState(false);
+  //const [selectedValue, setSelectedValue] = useState(false);
 
-  	// Used to swap from login to register. True = login, false = register
+  // Used to swap from login to register. True = login, false = register
-  	const [loginCheck, setLoginCheck] = useState(true);
+  const [loginCheck, setLoginCheck] = useState(true);
 
-	// Error messages etc
+  // Error messages etc
-	const [loginInfo, setLoginInfo] = useState("");
+  const [loginInfo, setLoginInfo] = useState("");
 
-	const handleValidateForm = () => {
+  const handleValidateForm = () => {
-		return (username.length > 1 && password.length > 8);
+    return username.length > 1 && password.length > 8;
-	}
+  };
 
-	const onSubmit = (e) => {
+  const onSubmit = (e) => {
-		e.preventDefault()
+    e.preventDefault();
 
-		// Just use this one?
+    // Just use this one?
-		var data = '{"username": "' + username + '", "password": "' + password + '"}';
+    var data =
-		var baseurl = globalUrl
+      '{"username": "' + username + '", "password": "' + password + '"}';
-		if (loginCheck) {
+    var baseurl = globalUrl;
-			var url = baseurl+'/login';
+    if (loginCheck) {
-			fetch(url, {
+      var url = baseurl + "/login";
-				method: 'POST',
+      fetch(url, {
-				body: data,
+        method: "POST",
-				headers: {
+        body: data,
-					'Content-Type': 'application/json',
+        headers: {
-				},
+          "Content-Type": "application/json",
-			})
+        },
-			.then(response =>
+      })
-				response.json().then(responseJson => {
+        .then((response) =>
-					console.log(responseJson)
+          response.json().then((responseJson) => {
-					//console.log(e)
+            console.log(responseJson);
-					if (responseJson["success"] === false) {
+            //console.log(e)
-						setLoginInfo(responseJson["reason"])
+            if (responseJson["success"] === false) {
-					} else {
+              setLoginInfo(responseJson["reason"]);
-						setLoginInfo("Successful login :)")
+            } else {
-						onClose()
+              setLoginInfo("Successful login :)");
-						setIsLoggedIn(true)
+              onClose();
-					}
+              setIsLoggedIn(true);
-				}),
+            }
-			)
+          })
-			.catch(error => {
+        )
-				setLoginInfo("Error in userdata")
+        .catch((error) => {
-			});
+          setLoginInfo("Error in userdata");
-		} else {
+        });
-			url = baseurl+'/register';
+    } else {
-			fetch(url, {
+      url = baseurl + "/register";
-				method: 'POST',
+      fetch(url, {
-				body: data,
+        method: "POST",
-				headers: {
+        body: data,
-					'Content-Type': 'application/json',
+        headers: {
-				},
+          "Content-Type": "application/json",
-			})
+        },
-			.then(response =>
+      })
-				response.json().then(responseJson => {
+        .then((response) =>
-					if (responseJson["success"] === false) {
+          response.json().then((responseJson) => {
-						setLoginInfo(responseJson["reason"])
+            if (responseJson["success"] === false) {
-					} else {
+              setLoginInfo(responseJson["reason"]);
-						setLoginInfo("Successful register :)")
+            } else {
-						onClose()
+              setLoginInfo("Successful register :)");
-						setIsLoggedIn(true)
+              onClose();
-					}
+              setIsLoggedIn(true);
-				}),
+            }
-			)
+          })
-			.catch(error => {
+        )
-				setLoginInfo("Error in userdata")
+        .catch((error) => {
-			});
+          setLoginInfo("Error in userdata");
-		}
+        });
-	}
+    }
+  };
 
-	const onChangeUser = (e) => {
+  const onChangeUser = (e) => {
-		setUsername(e.target.value)	
+    setUsername(e.target.value);
-	}
+  };
 
-	const onChangePass = (e) => {
+  const onChangePass = (e) => {
-		setPassword(e.target.value)	
+    setPassword(e.target.value);
-	}
+  };
 
-	const onClickRegister = () => {
+  const onClickRegister = () => {
-		setLoginCheck(!loginCheck)
+    setLoginCheck(!loginCheck);
-	}
+  };
 
-	//var loginChange = loginCheck ? (<div><p onClick={setLoginCheck(false)}>Want to register? Click here.</p></div>) : (<div><p onClick={setLoginCheck(true)}>Go back to login? Click here.</p></div>);
+  //var loginChange = loginCheck ? (<div><p onClick={setLoginCheck(false)}>Want to register? Click here.</p></div>) : (<div><p onClick={setLoginCheck(true)}>Go back to login? Click here.</p></div>);
-	var formtitle = loginCheck ? <div>Login</div> : <div>Register</div>
+  var formtitle = loginCheck ? <div>Login</div> : <div>Register</div>;
-	var formButton = loginCheck ? <div>Click to Register</div> : <div>Click to Login</div>
+  var formButton = loginCheck ? (
+    <div>Click to Register</div>
+  ) : (
+    <div>Click to Login</div>
+  );
 
-	return (
+  return (
-		<Dialog modal open={open} onClose={onClose} {...other}>
+    <Dialog modal open={open} onClose={onClose} {...other}>
-			<DialogTitle>{formtitle}</DialogTitle>
+      <DialogTitle>{formtitle}</DialogTitle>
-      <form onSubmit={onSubmit} style={{margin: "15px 15px 15px 15px"}}>
+      <form onSubmit={onSubmit} style={{ margin: "15px 15px 15px 15px" }}>
-			Username
+        Username
-			<div>
+        <div>
-				<TextField
+          <TextField
-					required
+            required
-				  id="standard-required"
+            id="standard-required"
-					autoComplete="username"
+            autoComplete="username"
-					margin="normal"
+            margin="normal"
-					variant="outlined"
+            variant="outlined"
-      	 	onChange={onChangeUser}
+            onChange={onChangeUser}
-				/>
+          />
-			</div>
+        </div>
-			Password	
+        Password
-			<div>
+        <div>
-				<TextField
+          <TextField
-					id="outlined-password-input"
+            id="outlined-password-input"
-					type="password"
+            type="password"
-					autoComplete="current-password"
+            autoComplete="current-password"
-					margin="normal"
+            margin="normal"
-					variant="outlined"
+            variant="outlined"
-      	 	onChange={onChangePass}
+            onChange={onChangePass}
-				/>
+          />
-			</div>
+        </div>
-			<div style={{display: "flex", marginTop: "15px"}}>
+        <div style={{ display: "flex", marginTop: "15px" }}>
-				<Button color="secondary" variant="contained" type="submit" style={{flex: "1", marginRight: "5px"}} disabled={!handleValidateForm()}>SUBMIT</Button> 				
+          <Button
-				
+            color="secondary"
-				<Button color="primary" variant="contained" type="button" style={{flex: "1"}} onClick={onClose}>Cancel</Button> 				
+            variant="contained"
-			</div>
+            type="submit"
-			{loginInfo}
+            style={{ flex: "1", marginRight: "5px" }}
-			</form>
+            disabled={!handleValidateForm()}
-			<div style={{display: "flex"}}>
+          >
-				<Button color="secondary" variant="contained" onClick={onClickRegister} type="button" style={{flex: "1"}}>{formButton}</Button> 				
+            SUBMIT
-			</div>
+          </Button>
-		</Dialog>
+
-	);
+          <Button
-}
+            color="primary"
+            variant="contained"
+            type="button"
+            style={{ flex: "1" }}
+            onClick={onClose}
+          >
+            Cancel
+          </Button>
+        </div>
+        {loginInfo}
+      </form>
+      <div style={{ display: "flex" }}>
+        <Button
+          color="secondary"
+          variant="contained"
+          onClick={onClickRegister}
+          type="button"
+          style={{ flex: "1" }}
+        >
+          {formButton}
+        </Button>
+      </div>
+    </Dialog>
+  );
+};
 
 export default LoginDialog;