This commit is contained in:
frikky
2021-04-22 17:49:42 +02:00
75 changed files with 16975 additions and 12015 deletions
-15
View File
@@ -1,15 +0,0 @@
**/.git*
**/.gitlab-ci.yml
**/.env
**/.dockerignore
**/Dockerfile*
**/node_modules
buildSrc/libs
.gradle/
build/
**build/
.idea/
!.idea/codeStyles/codeStyleConfig.xml
.DS_Store
*.log
out/
+5 -1
View File
@@ -12,6 +12,7 @@ SHUFFLE_APP_DOWNLOAD_LOCATION=https://github.com/frikky/shuffle-apps
SHUFFLE_DOWNLOAD_AUTH_USERNAME=
SHUFFLE_DOWNLOAD_AUTH_PASSWORD=
SHUFFLE_DOWNLOAD_AUTH_BRANCH=
SHUFFLE_APP_FORCE_UPDATE=false
# User config for first load. Username & PW: min length 3
SHUFFLE_DEFAULT_USERNAME=
@@ -39,4 +40,7 @@ SHUFFLE_PASS_WORKER_PROXY=TRUE
SHUFFLE_BASE_IMAGE_REGISTRY=ghcr.io
SHUFFLE_BASE_IMAGE_NAME=frikky
SHUFFLE_BASE_IMAGE_TAG_SUFFIX="-0.8.0"
SHUFFLE_BASE_IMAGE_TAG_SUFFIX="-0.8.60"
# Used for auto-cleanup of containers. REALLY important at scale.
SHUFFLE_CONTAINER_AUTO_CLEANUP=false
+39
View File
@@ -0,0 +1,39 @@
# Contributing to Shuffle
First off, thank you for contributing to [Shuffle](https://shuffler.io/)! Your talents and your contributions are greatly appreciated. With Shuffle, we aim to make cybersecurity more accessible, and keep that in mind with everything we make.
## Opening a new issue
If you find a bug or think of an improvement or fix, please open a [new issue](https://github.com/frikky/Shuffle/issues/new). Outline every step necessary to reproduce the bug. Include screenshots, logs and/or code examples where applicable. The more thorough you are, the better.
## What you can work on
There are a lot of things to work on in our complex ecosystem. The most pressing issues are documentation, use-cases and content-creation, but any help is appreciated. We'll make sure you get the help you need to get started. Below is an incomplete list of items. If you see an issue, tell us or fix it! :)
#### App Creation (Python & GUI w/OpenAPI)
As with everything else, app creation for Shuffle is made as accessibl as possible with the app editor. However, there are some instances where it can't do the job, and you'll have to write Python code. The App Editor generates OpenAPI specifications and can be widely shared, while Python apps only work for Shuffle and NSA's WALKOFF (which Shuffle is based on). You can find our [OpenAPI apps here](https://github.com/frikky/security-openapis) and our [Python apps here](https://github.com/frikky/shuffle-apps). Apps in these repositories are automatically available after installation. Shuffle apps are searchable on [https://shuffler.io](https://shuffler.io/search).
#### Workflow creation (GUI & Conceptualizing)
Workflows are where the magic of Shuffle automation happens. Our current ones [are outlined here](https://github.com/frikky/security-openapis), and will be automatically imported into Shuffle instances in the future. They are split into Prepare and Response, but don't necessarily have to be. If you'd like to talk about workflow creation or use-cases in general, either Open a [new issue](https://github.com/frikky/shuffle-workflows/issues/new) or send us an email at [frikky@shuffler.io](mailto:frikky@shuffler.io)
#### Documentation (Markdown)
Documentation is essential to any product, and Shuffle is no exception. Documentation in Shuffle uses markdown and is located in the [shuffle-docs](https://github.com/frikky/shuffle-docs/tree/master/docs) repository. These are then loaded into Shuffle when someone visits [https://shuffler/docs/about](https://shuffler/docs/about), then cached for later use. If you make an edit, expect it on our website in about an hour.
#### Frontend (ReactJS)
The frontend of Shuffle is what everyone sees when they log in. Our goal here is to make it easy to get started and keep going with Shuffle - removing any blockers from the point of accessibility. If you'd like to get started, find [an issue](https://github.com/frikky/Shuffle/issues) and check the [installation guide](https://github.com/frikky/Shuffle/blob/master/install-guide.md#local-development-installation) for setting it up locally without Docker.
#### Backend (Golang)
The backend of Shuffle is our REST API Server that runs in the background, handling all the API-calls in general, whether from users or apps. If you'd like to get started, find [an issue](https://github.com/frikky/Shuffle/issues) and check the [installation guide](https://github.com/frikky/Shuffle/blob/master/install-guide.md#local-development-installation) for setting it up locally without Docker.
## Working on an issue
**Shuffle** uses the [GitHub flow](https://guides.github.com/introduction/flow/index.html). All project changes are made through pull requests.
If you see an issue that you would like to work on, leave a quick comment or just get cracking.
### License
All contributions are made under either the **GNU Affero General Public License v3.0** or **MIT** license. See below for further details.
* [Main project license - AGPLv3](https://github.com/frikky/Shuffle/blob/master/LICENSE)
* [Apps - MIT](https://github.com/frikky/Shuffle-apps/blob/master/LICENSE)
* [Workflows - MIT](https://github.com/frikky/Shuffle-workflows/blob/master/LICENSE)
* [Documentation - MIT](https://github.com/frikky/Shuffle-docs/blob/master/LICENSE)
+2 -2
View File
@@ -2,8 +2,8 @@
github: frikky
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
open_collective: shuffle
ko_fi: frikky
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
+17 -8
View File
@@ -82,8 +82,13 @@ docker-compose up
Related issue: #47
## Local development installation
**Frontend - ReactJS /w cytoscape**
# Local development installation
Local development is pretty straight forward with **ReactJS** and **Golang**. This part is intended to help you run the code for development purposes.
**PS: You have to stop the Backend Docker container to get this one working**
**PPS: Use the "Launch" branch when developing to get it set up easier**
## Frontend - ReactJS /w cytoscape
http://localhost:3000 - Requires [npm](https://nodejs.org/en/download/)/[yarn](https://yarnpkg.com/lang/en/docs/install/#debian-stable)/your preferred manager. Runs independently from backend.
```bash
cd frontend
@@ -91,35 +96,39 @@ npm i
npm start
```
**Backend - Golang**
## Backend - Golang
http://localhost:5001 - REST API - requires [>=go1.13](https://golang.org/dl/)
```bash
export DATASTORE_EMULATOR_HOST=0.0.0.0:8000
cd backend/go-app
go build
go run *.go
```
**Database - Datastore**
**WINDOWS USERS:** You'll have to to add the "export" part as an environment variable.
## Database - Datastore
Based on Google datastore
```
docker run -p 8000:8000 google/cloud-sdk gcloud beta emulators datastore start --project=shuffle --host-port 0.0.0.0:8000 --no-store-on-disk
```
**Orborus**
## Orborus
Execution of Workflows:
PS: This requires some specific environment variables
```
cd functions/onprem/orborus
go run orborus.go
```
Environments:
Environments (modify for Windows):
```
export ORG_ID=Shuffle
export ENVIRONMENT_NAME=Shuffle
export BASE_URL=http://YOUR-IP:5001
export DOCKER_API_VERSION=1.40
export SHUFFLE_PASS_WORKER_PROXY=${SHUFFLE_PASS_WORKER_PROXY}
```
**WINDOWS USERS:** You'll have to to add the "export" part as an environment variable.
AND THAT's it - hopefully it worked. If it didn't please email [frikky@shuffler.io](mailto:frikky@shuffler.io)
+67
View File
@@ -0,0 +1,67 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"
on:
push:
branches: [ master ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ master ]
schedule:
- cron: '38 16 * * 4'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
language: [ 'go', 'javascript', 'python' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ]
# Learn more:
# https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#changing-the-languages-that-are-analyzed
steps:
- name: Checkout repository
uses: actions/checkout@v2
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1
# ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language
#- run: |
# make bootstrap
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
+4
View File
@@ -18,3 +18,7 @@ functions/generated_apps
backend/onprem/app_sdk/apps
*test.py
shuffle-database
*.exe
*debug*
+51 -23
View File
@@ -1,20 +1,22 @@
# Shuffle
[Shuffle](https://shuffler.io) is an automation platform to unify your security services (SOAR). It has thousands of premade integrations and is based on open frameworks like OpenAPI and Mitre Att&ck. The workflow editor is based on a no-code thought process to empower non-developers, and the app creator makes you able to integrate any platform in minutes.
[Shuffle](https://shuffler.io) is an automation platform focused on accessibility. We believe everyone should have access to efficient processes, and are striving to make that a possibility by making integrations for YOUR tools. Security Operations is complex, but it doesn't have to be.
[![Discord](https://img.shields.io/discord/463752820026376202.svg?label=&logo=discord&logoColor=ffffff&color=7389D8&labelColor=6A7EC2)](https://discord.gg/B2CBzUm)
![Example Shuffle webhook integration](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_webhook.png)
![Example Shuffle webhook integration](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/github_shuffle_img.png)
## Try it
* Self-hosted: Check out the [installation guide](https://github.com/frikky/shuffle/blob/master/install-guide.md)
* Self-hosted: Check out the [installation guide](https://github.com/frikky/shuffle/blob/master/.github/install-guide.md)
* Cloud: Register at https://shuffler.io/register and get cooking (missing a lot of features)
Please consider [sponsoring](https://github.com/sponsors/frikky) the project if you want to see more rapid development.
## Support
* [Discord](https://discord.gg/B2CBzUm)
* [Twitter](https://twitter.com/shuffleio)
* [Email](mailto:frikky@shuffler.io)
* [Open issue](https://github.com/frikky/Shuffle/issues/new)
* [Shuffler.io](https://shuffler.io/contact)
## Blogposts
* [1. Introducing Shuffle](https://medium.com/security-operation-capybara/introducing-shuffle-an-open-source-soar-platform-part-1-58a529de7d12)
@@ -23,31 +25,60 @@ Please consider [sponsoring](https://github.com/sponsors/frikky) the project if
* [4. Real-time executions with TheHive, Cortex and MISP](https://medium.com/@Frikkylikeme/indicators-and-webhooks-with-thehive-cortex-and-misp-open-source-soar-part-4-f70cde942e59)
## Documentation
[Documentation](https://shuffler.io/docs) can be found on https://shuffler.io/docs and is written in https://github.com/frikky/shuffle-docs.
[Documentation](https://shuffler.io/docs) can be found on [https://shuffler.io/docs](https://shuffler.io/docs) and is written here: [https://github.com/frikky/shuffle-docs](https://github.com/frikky/shuffle-docs).
## Related repositories
* Apps: https://github.com/frikky/shuffle-apps
* Workflows: https://github.com/frikky/shuffle-workflows
* Security OpenAPI apps: https://github.com/frikky/security-openapis
* Documentation: https://github.com/frikky/shuffle-docs
* OpenAPI apps: [https://github.com/frikky/security-openapis](https://github.com/frikky/security-openapis)
* Documentation: [https://github.com/frikky/shuffle-docs](https://github.com/frikky/shuffle-docs)
* Workflows: [https://github.com/frikky/shuffle-workflows](https://github.com/frikky/shuffle-workflows)
* Python apps: [https://github.com/frikky/shuffle-apps](https://github.com/frikky/shuffle-apps)
## Features
* Simple workflow automation editor
* Premade apps for a number of security tools
* App creator for [OpenAPI](https://github.com/frikky/OpenAPI-security-definitions)
* Easy to learn Python library for custom apps
## Architecture
![Shuffle Architecture](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_architecture.png)
* Simple, feature rich [workflow editor](https://shuffler.io/docs/workflows)
* App creator using [OpenAPI](https://github.com/frikky/OpenAPI-security-definitions)
* Premade apps for your security tools
* Organization and sub-organization control
* Hybrid resource sharing with shuffler.io (optional)
## Website
https://shuffler.io
[https://shuffler.io](https://shuffler.io)
## Contributing
We want to make the world of cybersecurity more accessible and need all the help we can get. Send an email to [frikky@shuffler](mailto:frikky@shuffler.io) and we'll make sure to give you any training you may need.
These are the main areas to contribute in:
* Frontend (ReactJS)
* Backend (Golang)
* App Creation (Python & GUI w/OpenAPI)
* Documentation (Markdown)
* Workflow creation (GUI & Conceptualizing)
* Content Creation (Blogs, videos etc)
Contributing guidelines are outlined [here](https://github.com/frikky/Shuffle/blob/master/.github/CONTRIBUTING.md).
## Contributors
![ICPL logo](https://github.com/frikky/Shuffle/blob/launch/frontend/src/assets/img/icpl_logo.png)
**Shuffle**
<a href="https://github.com/frikky/shuffle/graphs/contributors">
<img src="https://contrib.rocks/image?repo=frikky/shuffle" />
</a>
[**App magicians**](https://github.com/frikky/shuffle-apps)
<a href="https://github.com/frikky/shuffle-apps/graphs/contributors">
<img src="https://contrib.rocks/image?repo=frikky/shuffle-apps" />
</a>
## License
All modular information related to Shuffle will be under MIT (anyone can use it for whatever purpose), with Shuffle itself using AGPLv3.
Apps & App SDK: MIT
Workflows: MIT
Documentation: MIT
Shuffle backend: AGPLv3
Apps, specification and App SDK: MIT
## Architecture
![Shuffle Architecture](https://github.com/frikky/Shuffle/blob/master/frontend/src/assets/img/shuffle_architecture.png)
### Repository overview
Below is the folder structure with a short explanation
@@ -55,17 +86,14 @@ Below is the folder structure with a short explanation
├── README.md # What you're reading right now
├── backend # Contains backend related code.
│   ├── go-app # The backend golang webserver
│   ├── app_gen # Code for app generation outside the Shuffle platform
│ └── app_sdk # The SDK used for apps
├── frontend # Contains frontend code. ReactJS and cytoscape. Horrible code :)
├── functions # Contains google cloud function code mainly.
│   ├── static_baseline.py # Static code used by stitcher.go to generate code
│   ├── stitcher.go # Attempts to stitch together an app - part of backend now
├── frontend # Contains frontend code. ReactJS, Material UI and cytoscape
├── functions # Has execution and extension resources, such as the Wazuh integration
│   ├── onprem # Code for onprem solutions
│  │   ├── Orborus # Distributes execution locations
│  │   ├── Worker # Runs a workflow
└ docker-compose.yml # Used for deployments
```
**It's in BETA** - [Get in touch](https://shuffler.io/contact), send a mail to [frikky@shuffler.io](mailto:frikky@shuffler.io) or poke me on twitter [@frikkylikeme](https://twitter.com/frikkylikeme)
**It's in BETA (0.8.60)** - [Get in touch](https://shuffler.io/contact), send a mail to [frikky@shuffler.io](mailto:frikky@shuffler.io) or poke me on twitter [@frikkylikeme](https://twitter.com/frikkylikeme)
+2 -4
View File
@@ -1,4 +1,4 @@
from golang as builder
FROM golang:1.16.0-buster as builder
# Add files
RUN mkdir /app
@@ -7,14 +7,12 @@ WORKDIR /app
ADD ./go-app/main.go /app
ADD ./go-app/walkoff.go /app
ADD ./go-app/docker.go /app
ADD ./go-app/codegen.go /app
ADD ./go-app/files.go /app
ADD ./go-app/oauth2.go /app
ADD ./go-app/go.mod /app
# Required files for code generation
ADD ./app_sdk/app_base.py /app_sdk
ADD ./app_sdk/static_baseline.py /app_sdk
ADD ./app_sdk_kali/app_base.py /app_sdk_kali
ADD ./app_sdk_kali/static_baseline.py /app_sdk_kali
ADD ./app_sdk_blackarch/app_base.py /app_sdk_blackarch
+2 -2
View File
@@ -1,4 +1,4 @@
FROM python:3.7-alpine as base
FROM python:3.9.4-alpine as base
FROM base as builder
RUN apk --no-cache add --update alpine-sdk libffi libffi-dev musl-dev openssl-dev
@@ -7,7 +7,7 @@ RUN mkdir /install
WORKDIR /install
COPY requirements.txt /requirements.txt
RUN pip install --prefix="/install" -r /requirements.txt
RUN pip3 install -r /requirements.txt
FROM base
-4
View File
@@ -1,9 +1,5 @@
# app_sdk.py
This is the SDK used for apps to behave like they should.
To change it in the backend, upload it to Buckets/shuffler.appspot.com/generated_apps/baseline.
# static_baseline.py
It's used for python code generation and should be under MIT. Has to be located here because it's used by the backend.
## If you want to update apps.. PS: downloads from docker hub do overrides.. :)
1. Write your code & check if runtime works
+1118 -227
View File
File diff suppressed because it is too large Load Diff
+2 -1
View File
@@ -1,6 +1,6 @@
#!/bin/bash
NAME=shuffle-app_sdk
VERSION=0.8.2
VERSION=0.8.64
docker rmi docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION --force
docker build . -t frikky/shuffle:app_sdk -t frikky/$NAME:$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
@@ -8,6 +8,7 @@ docker build . -t frikky/shuffle:app_sdk -t frikky/$NAME:$VERSION -t docker.pkg.
#docker push frikky/$NAME:$VERSION
#docker push docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION
#docker push ghcr.io/frikky/$NAME:$VERSION
#docker tag ghcr.io/frikky/$NAME:$VERSION frikky/shuffle:app_sdk
docker push frikky/shuffle:app_sdk
docker push ghcr.io/frikky/$NAME:$VERSION
+2 -2
View File
@@ -1,2 +1,2 @@
requests
urllib3
urllib3=1.25.9
requests=2.25.1
-76
View File
@@ -1,76 +0,0 @@
import os
import sys
import time
import logging
import requests
# Goal here:
# * Make an app from WALKOFF able to run without app_base.py from WALKOFF
# # How:
# * Make it rely 100% on INPUT throug HTTP invocations instead of redis READS
# # But really, how?
# * Make a WORKER that reads the queue, and reuses a function
# Here to get it global
apikey = ""
try:
apikey = os.environ["FUNCTION_APIKEY"]
except KeyError:
pass
# Authorize the execution
def authorization(request):
# This is basically my issue, but it enforces the use of an internal API key for execution
try:
apikey = os.environ["FUNCTION_APIKEY"]
except KeyError:
return f"Internal server error", 500
# Check API key from ENV authentication
authentication = request.headers.get("Authorization")
if authentication == None: return f"Unauthorized", 401
apikey_split = authentication.split(" ")
if apikey_split[0] != "Bearer" or len(apikey_split) != 2:
return f"Apikey error", 401
if apikey != apikey_split[1]:
return f"Unauthorized", 401
return run(request)
class AppBase:
""" The base class for Python-based Walkoff applications, handles Redis and logging configurations. """
__version__ = None
app_name = None
def __init__(self, redis=None, logger=None, console_logger=None):#, docker_client=None):
self.logger = logger if logger is not None else logging.getLogger("AppBaseLogger")
self.redis=redis
self.console_logger=console_logger
self.current_execution_id = None
self.url = "https://shuffler.io"
self.apikey = apikey
@classmethod
async def run(cls, action):
""" Connect to Redis and HTTP session, await actions """
logging.basicConfig(format="{asctime} - {name} - {levelname}:{message}", style='{')
logger = logging.getLogger(f"{cls.__name__}")
logger.setLevel(logging.DEBUG)
app = cls(redis=None, logger=logger, console_logger=logger)
# Authorization for the app/function to control the workflow
# Function will crash if its wrong, which it probably should.
await app.execute_action(action)
async def execute_action(self, action):
# FIXME - add request for the function STARTING here. Use "results stream" or something
# PAUSED, AWAITING_DATA, PENDING, COMPLETED, ABORTED, EXECUTING, SUCCESS, FAILURE
self.authorization = action["authorization"]
self.execution_id = action["execution_id"]
self.current_execution_id = action["execution_id"]
-98
View File
@@ -1,98 +0,0 @@
runtime: go111
env_variables:
automatic_scaling:
max_instances: 1
min_instances: 1
handlers:
- url: /api/(.*)
script: auto
secure: always
- url: /static/js/(.*)
static_files: build/static/js/\1
upload: build/static/js/(.*)
secure: always
- url: /static/css/(.*)
static_files: build/static/css/\1
upload: build/static/css/(.*)
secure: always
- url: /images/(.*)
static_files: build/images/\1
upload: build/images/(.*)
secure: always
- url: /(.*\.(json|ico))$
static_files: build/\1
upload: build/.*\.(json|ico)$
secure: always
- url: /manifest.json
static_files: build/manifest.json
upload: build/manifest.json
secure: always
# lol.. wildcard doesn't work with /api/(.*) for some reason
- url: /
static_files: build/index.html
upload: build/index.html
secure: always
- url: /home
static_files: build/index.html
upload: build/index.html
secure: always
- url: /passwordreset
static_files: build/index.html
upload: build/index.html
secure: always
- url: /login
static_files: build/index.html
upload: build/index.html
secure: always
- url: /register
static_files: build/index.html
upload: build/index.html
secure: always
- url: /workflows
static_files: build/index.html
upload: build/index.html
secure: always
- url: /workflows/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
- url: /info/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
- url: /docs/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
- url: /docs
static_files: build/index.html
upload: build/index.html
secure: always
- url: /settings
static_files: build/index.html
upload: build/index.html
secure: always
- url: /apps
static_files: build/index.html
upload: build/index.html
secure: always
- url: /contact
static_files: build/index.html
upload: build/index.html
secure: always
- url: /apps/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
- url: /register/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
- url: /passwordreset/(.*)
static_files: build/index.html
upload: build/index.html
secure: always
File diff suppressed because it is too large Load Diff
+195 -11
View File
@@ -2,12 +2,15 @@ package main
// Docker
import (
"github.com/frikky/shuffle-shared"
"archive/tar"
"path/filepath"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
@@ -23,7 +26,6 @@ import (
"net/http"
"os"
"strings"
//"google.golang.org/appengine"
)
// Parses a directory with a Dockerfile into a tar for Docker images..
@@ -125,12 +127,29 @@ func getParsedTarMemory(fs billy.Filesystem, tw *tar.Writer, baseDir, extra stri
return err
}
//log.Printf("FILENAME: %s", filename)
readFile, err := ioutil.ReadAll(fileReader)
if err != nil {
log.Printf("Not file: %s", err)
return err
}
// Fixes issues with older versions of Docker and reference formats
// Specific to Shuffle rn. Could expand.
// FIXME: Seems like the issue was with multi-stage builds
/*
if filename == "Dockerfile" {
log.Printf("Should search and replace in readfile.")
referenceCheck := "FROM frikky/shuffle:"
if strings.Contains(string(readFile), referenceCheck) {
log.Printf("SHOULD SEARCH & REPLACE!")
newReference := fmt.Sprintf("FROM registry.hub.docker.com/frikky/shuffle:")
readFile = []byte(strings.Replace(string(readFile), referenceCheck, newReference, -1))
}
}
*/
//log.Printf("Filename: %s", filename)
// FIXME - might need the folder from EXTRA here
// Name has to be e.g. just "requirements.txt"
@@ -156,8 +175,23 @@ func getParsedTarMemory(fs billy.Filesystem, tw *tar.Writer, baseDir, extra stri
return nil
}
/*
// Fixes App SDK issues.. meh
func fixTags(tags []string) []string {
checkTag := "frikky/shuffle"
newTags := []string{}
for _, tag := range tags {
if strings.HasPrefix(tag, checkTags) {
newTags.append(newTags, fmt.Sprintf("registry.hub.docker.com/%s", tag))
}
newTags.append(tag)
}
}
*/
// Custom Docker image builder wrapper in memory
func buildImageMemory(fs billy.Filesystem, tags []string, dockerfileFolder string) error {
func buildImageMemory(fs billy.Filesystem, tags []string, dockerfileFolder string, downloadIfFail bool) error {
ctx := context.Background()
client, err := client.NewEnvClient()
if err != nil {
@@ -169,7 +203,7 @@ func buildImageMemory(fs billy.Filesystem, tags []string, dockerfileFolder strin
tw := tar.NewWriter(buf)
defer tw.Close()
log.Printf("Setting up memory build structure for folder: %s", dockerfileFolder)
log.Printf("[INFO] Setting up memory build structure for folder: %s", dockerfileFolder)
err = getParsedTarMemory(fs, tw, dockerfileFolder, "")
if err != nil {
log.Printf("Tar issue: %s", err)
@@ -197,17 +231,56 @@ func buildImageMemory(fs billy.Filesystem, tags []string, dockerfileFolder strin
}
// Build the actual image
log.Printf("[INFO] Building %s. This may take up to a few minutes.", dockerfileFolder)
imageBuildResponse, err := client.ImageBuild(
ctx,
dockerFileTarReader,
buildOptions,
)
//log.Printf("Response: %#v", imageBuildResponse.Body)
//log.Printf("IMAGERESPONSE: %#v", imageBuildResponse.Body)
defer imageBuildResponse.Body.Close()
_, newerr := io.Copy(os.Stdout, imageBuildResponse.Body)
buildBuf := new(strings.Builder)
_, newerr := io.Copy(buildBuf, imageBuildResponse.Body)
if newerr != nil {
log.Printf("Failed reading Docker build STDOUT: %s", newerr)
} else {
log.Printf("STRING: %s", buildBuf.String())
if strings.Contains(buildBuf.String(), "errorDetail") {
log.Printf("[ERROR] Docker build:\n%s\nERROR ABOVE: Trying to pull tags from: %s", buildBuf.String(), strings.Join(tags, "\n"))
// Handles pulling of the same image if applicable
// This fixes some issues with older versions of Docker which can't build
// on their own ( <17.05 )
pullOptions := types.ImagePullOptions{}
downloaded := false
for _, image := range tags {
// Is this ok? Not sure. Tags shouldn't be controlled here prolly.
image = strings.ToLower(image)
newImage := fmt.Sprintf("%s/%s", registryName, image)
log.Printf("[INFO] Pulling image %s", newImage)
reader, err := client.ImagePull(ctx, newImage, pullOptions)
if err != nil {
log.Printf("[ERROR] Failed getting image %s: %s", newImage, err)
continue
}
// Attempt to retag the image to not contain registry...
//newBuf := buildBuf
downloaded = true
io.Copy(os.Stdout, reader)
log.Printf("[INFO] Successfully downloaded and built %s", newImage)
}
if !downloaded {
return errors.New(fmt.Sprintf("Failed to build / download images %s", strings.Join(tags, ",")))
}
//baseDockerName
}
}
if err != nil {
@@ -227,7 +300,7 @@ func buildImage(tags []string, dockerfileFolder string) error {
return err
}
log.Printf("Tags: %s", tags)
log.Printf("[INFO] Docker Tags: %s", tags)
dockerfileSplit := strings.Split(dockerfileFolder, "/")
// Create a buffer
@@ -272,9 +345,15 @@ func buildImage(tags []string, dockerfileFolder string) error {
// Read the STDOUT from the build process
defer imageBuildResponse.Body.Close()
_, err = io.Copy(os.Stdout, imageBuildResponse.Body)
buildBuf := new(strings.Builder)
_, err = io.Copy(buildBuf, imageBuildResponse.Body)
if err != nil {
return err
} else {
if strings.Contains(buildBuf.String(), "errorDetail") {
log.Printf("[ERROR] Docker build:\n%s\nERROR ABOVE: Trying to pull tags from: %s", buildBuf.String(), strings.Join(tags, "\n"))
return errors.New(fmt.Sprintf("Failed building %s. Check backend logs for details. Most likely means you have an old version of Docker.", strings.Join(tags, ",")))
}
}
return nil
@@ -424,7 +503,7 @@ func handleStopHookDocker(resp http.ResponseWriter, request *http.Request) {
ctx := context.Background()
hook, err := getHook(ctx, fileId)
if err != nil {
log.Printf("Failed getting hook: %s", err)
log.Printf("Failed getting hook %s (stop docker): %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
@@ -556,7 +635,7 @@ func handleStartHookDocker(resp http.ResponseWriter, request *http.Request) {
ctx := context.Background()
hook, err := getHook(ctx, fileId)
if err != nil {
log.Printf("Failed getting hook: %s", err)
log.Printf("Failed getting hook %s (start docker): %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
@@ -636,7 +715,7 @@ func handleStartHookDocker(resp http.ResponseWriter, request *http.Request) {
}
// FIXME - get some real data?
log.Printf("Successfully started %s-%s on port %s with filepath %s", image, fileId, port, filepath)
log.Printf("[INFO] Successfully started %s-%s on port %s with filepath %s", image, fileId, port, filepath)
resp.WriteHeader(200)
resp.Write([]byte(`{"success": true, "message": "Started webhook"}`))
return
@@ -644,7 +723,7 @@ func handleStartHookDocker(resp http.ResponseWriter, request *http.Request) {
// Checks if an image exists
func imageCheckBuilder(images []string) error {
log.Printf("[FIXME] ImageNames to check: %#v", images)
//log.Printf("[FIXME] ImageNames to check: %#v", images)
return nil
ctx := context.Background()
@@ -704,10 +783,115 @@ func hookTest() {
returnHook, err := getHook(ctx, hook.Id)
if err != nil {
log.Printf("Failed getting hook: %s", err)
log.Printf("Failed getting hook %s (test): %s", hook.Id, err)
}
if len(returnHook.Id) > 0 {
log.Printf("Success! - %s", returnHook.Id)
}
}
//https://stackoverflow.com/questions/23935141/how-to-copy-docker-images-from-one-host-to-another-without-using-a-repository
func getDockerImage(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
// Just here to verify that the user is logged in
_, err := shuffle.HandleApiAuthentication(resp, request)
if err != nil {
log.Printf("Api authentication failed in validate swagger: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
body, err := ioutil.ReadAll(request.Body)
if err != nil {
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Failed reading body"}`))
return
}
type requestCheck struct {
Name string `datastore:"name" json:"name" yaml:"name"`
}
//body = []byte(`swagger: "2.0"`)
//body = []byte(`swagger: '1.0'`)
//newbody := string(body)
//newbody = strings.TrimSpace(newbody)
//body = []byte(newbody)
//log.Println(string(body))
//tmpbody, err := yaml.YAMLToJSON(body)
//log.Println(err)
//log.Println(string(tmpbody))
// This has to be done in a weird way because Datastore doesn't
// support map[string]interface and similar (openapi3.Swagger)
var version requestCheck
err = json.Unmarshal(body, &version)
if err != nil {
resp.WriteHeader(422)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed JSON marshalling: %s"}`, err)))
return
}
log.Printf("Image to load: %s", version.Name)
//cli, err := client.NewEnvClient()
//if err != nil {
// log.Println("Unable to create docker client")
// return err
//}
dockercli, err := client.NewEnvClient()
if err != nil {
log.Printf("Unable to create docker client: %s", err)
resp.WriteHeader(422)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed JSON marshalling: %s"}`, err)))
return
}
ctx := context.Background()
images, err := dockercli.ImageList(ctx, types.ImageListOptions{
All: true,
})
img := types.ImageSummary{}
tagFound := ""
for _, image := range images {
for _, tag := range image.RepoTags {
log.Printf("[INFO] Docker Image: %s", tag)
if strings.ToLower(tag) == strings.ToLower(version.Name) {
img = image
tagFound = tag
break
}
}
}
if len(img.ID) == 0 {
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "message": "Couldn't find image %s"}`, version.Name)))
return
}
_ = tagFound
/*
log.Printf("IMg: %#v", img)
pullOptions := types.ImagePullOptions{}
log.Printf("[INFO] Pulling image %s", image)
reader, err := dockercli.ImagePull(ctx, tag, pullOptions)
if err != nil {
log.Printf("[ERROR] Failed getting image %s: %s", image, err)
}
io.Copy(os.Stdout, r)
*/
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"success": true, "message": "Downloading image %s"}`, version.Name)))
}
-764
View File
@@ -1,764 +0,0 @@
package main
/*
Handles files within Workflows.of Shuffle
*/
import (
"bytes"
"context"
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"strconv"
"strings"
"time"
"cloud.google.com/go/datastore"
"github.com/satori/go.uuid"
)
type File struct {
Id string `json:"id" datastore:"id"`
Type string `json:"type" datastore:"type"`
CreatedAt int64 `json:"created_at" datastore:"created_at"`
UpdatedAt int64 `json:"updated_at" datastore:"updated_at"`
MetaAccessAt int64 `json:"meta_access_at" datastore:"meta_access_at"`
DownloadAt int64 `json:"last_downloaded" datastore:"last_downloaded"`
Description string `json:"description" datastore:"description"`
ExpiresAt string `json:"expires_at" datastore:"expires_at"`
Status string `json:"status" datastore:"status"`
Filename string `json:"filename" datastore:"filename"`
URL string `json:"url" datastore:"org"`
OrgId string `json:"org_id" datastore:"org_id"`
WorkflowId string `json:"workflow_id" datastore:"workflow_id"`
Workflows []string `json:"workflows" datastore:"workflows"`
DownloadPath string `json:"download_path" datastore:"download_path"`
Md5sum string `json:"md5_sum" datastore:"md5_sum"`
Sha256sum string `json:"sha256_sum" datastore:"sha256_sum"`
}
var basepath = os.Getenv("SHUFFLE_FILE_LOCATION")
func fileAuthentication(request *http.Request) (string, error) {
executionId, ok := request.URL.Query()["execution_id"]
if ok && len(executionId) > 0 {
ctx := context.Background()
workflowExecution, err := getWorkflowExecution(ctx, executionId[0])
if err != nil {
log.Printf("[ERROR] Couldn't find execution ID %s", executionId[0])
return "", err
}
apikey := request.Header.Get("Authorization")
if !strings.HasPrefix(apikey, "Bearer ") {
log.Printf("[ERROR} Apikey doesn't start with bearer (2)")
return "", errors.New("No auth key found")
}
apikeyCheck := strings.Split(apikey, " ")
if len(apikeyCheck) != 2 {
log.Printf("[ERROR] Invalid format for apikey (2)")
return "", errors.New("No space in authkey")
}
// This is annoying af and is done because of maxlength lol
newApikey := apikeyCheck[1]
if newApikey != workflowExecution.Authorization {
//log.Printf("[ERROR] Bad apikey for execution %s. %s vs %s", executionId[0], apikey, workflowExecution.Authorization)
log.Printf("[ERROR] Bad apikey for execution %s.", executionId[0])
//%s vs %s", executionId[0], apikey, workflowExecution.Authorization)
return "", errors.New("Bad authorization key")
}
log.Printf("[INFO] Authorization is correct for execution %s!", executionId[0])
//%s vs %s. Setting Org", executionId, apikey, workflowExecution.Authorization)
if len(workflowExecution.ExecutionOrg) > 0 {
return workflowExecution.ExecutionOrg, nil
} else if len(workflowExecution.Workflow.ExecutingOrg.Id) > 0 {
return workflowExecution.ExecutionOrg, nil
} else {
log.Printf("[ERROR] Couldn't find org for workflow execution, but auth was correct.")
}
}
return "", errors.New("No execution id specified")
}
// https://golangcode.com/check-if-a-file-exists/
func fileExists(filename string) bool {
info, err := os.Stat(filename)
if os.IsNotExist(err) {
return false
}
return !info.IsDir()
}
func handleGetFileMeta(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
var fileId string
location := strings.Split(request.URL.String(), "/")
if location[1] == "api" {
if len(location) <= 4 {
log.Printf("[INFO] Path too short: %d", len(location))
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
fileId = location[4]
}
if strings.Contains(fileId, "?") {
fileId = strings.Split(fileId, "?")[0]
}
if len(fileId) != 36 {
log.Printf("Bad format for fileId %s", fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
return
}
log.Printf("\n\n[INFO] User is trying to GET File Meta for %s\n\n", fileId)
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("[INFO] INITIAL Api authentication failed in file deletion: %s", err)
orgId, err := fileAuthentication(request)
if err != nil {
log.Printf("[ERROR] Bad file authentication in get: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
user.ActiveOrg.Id = orgId
user.Username = "Execution File API"
}
// 1. Verify if the user has access to the file: org_id and workflow
log.Printf("[INFO] Should GET FILE META for %s if user has access", fileId)
ctx := context.Background()
file, err := getFile(ctx, fileId)
if err != nil {
log.Printf("[INFO] File %s not found: %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
found := false
if file.OrgId == user.ActiveOrg.Id {
found = true
} else {
for _, item := range user.Orgs {
if item == file.OrgId {
found = true
break
}
}
}
if !found {
log.Printf("[INFO] User %s doesn't have access to %s", user.Username, fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
newBody, err := json.Marshal(file)
if err != nil {
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false, "reason": "Failed to marshal filedata"}`))
return
}
log.Printf("[INFO] Successfully got file meta for %s", fileId)
resp.WriteHeader(200)
resp.Write([]byte(newBody))
}
func handleDeleteFile(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
var fileId string
location := strings.Split(request.URL.String(), "/")
if location[1] == "api" {
if len(location) <= 4 {
log.Printf("[INFO] Path too short: %d", len(location))
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
fileId = location[4]
}
if strings.Contains(fileId, "?") {
fileId = strings.Split(fileId, "?")[0]
}
if len(fileId) != 36 {
log.Printf("Bad format for fileId %s", fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
return
}
log.Printf("\n\n[INFO] User is trying to delete file %s\n\n", fileId)
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("[INFO] INITIAL Api authentication failed in file deletion: %s", err)
orgId, err := fileAuthentication(request)
if err != nil {
log.Printf("[ERROR] Bad file authentication in get: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
user.ActiveOrg.Id = orgId
user.Username = "Execution File API"
}
// 1. Verify if the user has access to the file: org_id and workflow
log.Printf("[INFO] Should DELETE file %s if user has access", fileId)
ctx := context.Background()
file, err := getFile(ctx, fileId)
if err != nil {
log.Printf("[INFO] File %s not found: %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
found := false
if file.OrgId == user.ActiveOrg.Id {
found = true
} else {
for _, item := range user.Orgs {
if item == file.OrgId {
found = true
break
}
}
}
if !found {
log.Printf("[INFO] User %s doesn't have access to %s", user.Username, fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
if file.Status == "deleted" {
log.Printf("[INFO] File with ID %s is already deleted.", fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
if fileExists(file.DownloadPath) {
err = os.Remove(file.DownloadPath)
if err != nil {
log.Printf("[ERROR] Failed deleting file locally: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed deleting filein path %s"}`, file.DownloadPath)))
return
}
log.Printf("[INFO] Deleted file %s locally. Next is database.", file.DownloadPath)
} else {
log.Printf("[ERROR] File doesn't exist. Can't delete. Should maybe delete file anyway?")
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "File in location %s doesn't exist"}`, file.DownloadPath)))
return
}
file.Status = "deleted"
err = setFile(ctx, *file)
if err != nil {
log.Printf("[ERROR] Failed setting file to deleted")
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false, "reason": "Failed setting file to deleted"}`))
return
}
/*
//Actually delete it?
err = DeleteKey(ctx, "files", fileId)
if err != nil {
log.Printf("Failed deleting file with ID %s: %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
*/
log.Printf("[INFO] Successfully deleted file %s", fileId)
resp.WriteHeader(200)
resp.Write([]byte(`{"success": true}`))
}
func handleGetFileContent(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
var fileId string
location := strings.Split(request.URL.String(), "/")
if location[1] == "api" {
if len(location) <= 4 {
log.Printf("Path too short: %d", len(location))
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
fileId = location[4]
}
if len(fileId) != 36 {
log.Printf("Bad format for fileId %s", fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
return
}
log.Printf("\n\nUser is trying to download file %s\n\n", fileId)
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("INITIAL Api authentication failed in file download: %s", err)
orgId, err := fileAuthentication(request)
if err != nil {
log.Printf("Bad file authentication in get: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
user.ActiveOrg.Id = orgId
user.Username = "Execution File API"
/*
} else {
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
*/
}
// 1. Verify if the user has access to the file: org_id and workflow
log.Printf("[INFO] Should get file %s", fileId)
ctx := context.Background()
file, err := getFile(ctx, fileId)
if err != nil {
log.Printf("[ERROR] File %s not found: %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
found := false
if file.OrgId == user.ActiveOrg.Id {
found = true
} else {
for _, item := range user.Orgs {
if item == file.OrgId {
found = true
break
}
}
}
if !found {
log.Printf("User %s doesn't have access to %s", user.Username, fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
if file.Status != "active" {
log.Printf("[ERROR] File status isn't active, but %s. Can't continue.", file.Status)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "The file isn't ready to be downloaded yet. Status required: active"}`))
return
}
// Fixme: More auth: org and workflow!
downloadPath := file.DownloadPath
log.Printf("[INFO] Downloadpath: %s", downloadPath)
Openfile, err := os.Open(downloadPath)
defer Openfile.Close() //Close after function return
if err != nil {
//File not found, send 404
http.Error(resp, "File not found.", 404)
return
}
//File is found, create and send the correct headers
//Get the Content-Type of the file
//Create a buffer to store the header of the file in
FileHeader := make([]byte, 512)
//Copy the headers into the FileHeader buffer
Openfile.Read(FileHeader)
//Get content type of file
FileContentType := http.DetectContentType(FileHeader)
//Get the file size
FileStat, _ := Openfile.Stat() //Get info from file
FileSize := strconv.FormatInt(FileStat.Size(), 10) //Get file size as a string
//Send the headers
resp.Header().Set("Content-Disposition", "attachment; filename="+fileId)
resp.Header().Set("Content-Type", FileContentType)
resp.Header().Set("Content-Length", FileSize)
//Send the file
//We read 512 bytes from the file already, so we reset the offset back to 0
Openfile.Seek(0, 0)
io.Copy(resp, Openfile) //'Copy' the file to the client
return
//log.Printf("Should download file %s", downloadPath)
}
func handleUploadFile(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
var fileId string
location := strings.Split(request.URL.String(), "/")
if location[1] == "api" {
if len(location) <= 4 {
log.Printf("Path too short: %d", len(location))
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
fileId = location[4]
}
if len(fileId) != 36 {
log.Printf("Bad format for fileId %s", fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Badly formatted fileId"}`))
return
}
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("INITIAL Api authentication failed in file upload: %s", err)
orgId, err := fileAuthentication(request)
if err != nil {
log.Printf("Bad file authentication in create file: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
user.ActiveOrg.Id = orgId
user.Username = "Execution File API"
}
log.Printf("[INFO] Should UPLOAD file %s if user has access", fileId)
ctx := context.Background()
file, err := getFile(ctx, fileId)
if err != nil {
log.Printf("File %s not found: %s", fileId, err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
found := false
if file.OrgId == user.ActiveOrg.Id {
found = true
} else {
for _, item := range user.Orgs {
if item == file.OrgId {
found = true
break
}
}
}
if !found {
log.Printf("User %s doesn't have access to %s", user.Username, fileId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
log.Printf("[INFO] STATUS: %s", file.Status)
if file.Status != "created" {
log.Printf("File status isn't created. Can't upload.")
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "This file already has data."}`))
return
}
request.ParseMultipartForm(32 << 20)
parsedFile, _, err := request.FormFile("shuffle_file")
if err != nil {
log.Printf("[ERROR] Couldn't upload file: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Failed uploading file"}`))
return
}
defer parsedFile.Close()
file.Status = "uploading"
err = setFile(ctx, *file)
if err != nil {
log.Printf("Failed setting file to uploading")
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false, "reason": "Failed setting file to uploading"}`))
return
}
// Can be used for validation files for change
var buf bytes.Buffer
io.Copy(&buf, parsedFile)
contents := buf.Bytes()
md5 := md5sum(contents)
buf.Reset()
sha256Sum := sha256.Sum256(contents)
//parsedFile.Reset()
f, err := os.OpenFile(file.DownloadPath, os.O_WRONLY|os.O_CREATE, os.ModePerm)
if err != nil {
// Rolling back file
file.Status = "created"
setFile(ctx, *file)
log.Printf("[ERROR] Failed uploading and creating file: %s", err)
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false}`))
return
}
defer f.Close()
parsedFile.Seek(0, io.SeekStart)
io.Copy(f, parsedFile)
// FIXME: Set this one to 200 anyway? Can't download file then tho..
file.Status = "active"
file.Md5sum = md5
file.Sha256sum = fmt.Sprintf("%x", sha256Sum)
log.Printf("[INFO] MD5 for file %s (%s) is %s and SHA256 is %s", file.Filename, file.Id, file.Md5sum, file.Sha256sum)
err = setFile(ctx, *file)
if err != nil {
log.Printf("[ERROR] Failed setting file back to active")
resp.WriteHeader(500)
resp.Write([]byte(`{"success": false, "reason": "Failed setting file to active"}`))
return
}
log.Printf("[INFO] Successfully uploaded file ID %s", file.Id)
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"success": true}`)))
}
func handleCreateFile(resp http.ResponseWriter, request *http.Request) {
cors := handleCors(resp, request)
if cors {
return
}
// 1. Check user directly
// 2. Check workflow execution authorization
user, err := handleApiAuthentication(resp, request)
if err != nil {
log.Printf("[INFO] INITIAL Api authentication failed in file creation: %s", err)
orgId, err := fileAuthentication(request)
if err != nil {
log.Printf("[ERROR] Bad file authentication in create file: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false}`))
return
}
user.ActiveOrg.Id = orgId
user.Username = "Execution File API"
}
body, err := ioutil.ReadAll(request.Body)
if err != nil {
log.Println("Failed reading body")
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed to read data"}`)))
return
}
type FileStructure struct {
Filename string `json:"filename"`
OrgId string `json:"org_id"`
WorkflowId string `json:"workflow_id"`
}
var curfile FileStructure
err = json.Unmarshal(body, &curfile)
if err != nil {
log.Printf("[ERROR] Failed unmarshaling: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed to unmarshal data"}`)))
return
}
// Loads of validation below
if len(curfile.Filename) == 0 || len(curfile.OrgId) == 0 || len(curfile.WorkflowId) == 0 {
log.Printf("[ERROR] Missing field during upload.")
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Missing field. Required: filename, org_id, workflow_id"}`)))
return
}
ctx := context.Background()
if user.ActiveOrg.Id != curfile.OrgId {
log.Printf("[ERROR] User can't access org %s", curfile.OrgId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Error with organization"}`))
return
}
// Try to get the org and workflow in case they don't exist
workflow, err := getWorkflow(ctx, curfile.WorkflowId)
if err != nil {
log.Printf("[ERROR] Workflow %s doesn't exist.", curfile.WorkflowId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Error with workflow id or org id"}`))
return
}
_, err = getOrg(ctx, curfile.OrgId)
if err != nil {
log.Printf("[ERROR] Org %s doesn't exist.", curfile.OrgId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Error with workflow id or org id"}`))
return
}
if workflow.ExecutingOrg.Id != curfile.OrgId {
found := false
for _, curorg := range workflow.Org {
if curorg.Id == curfile.OrgId {
found = true
break
}
}
if !found {
log.Printf("[ERROR] Org %s doesn't have access to %s.", curfile.OrgId, curfile.WorkflowId)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Error with workflow id or org id"}`))
return
}
}
if strings.Contains(curfile.Filename, "/") || strings.Contains(curfile.Filename, `"`) || strings.Contains(curfile.Filename, "..") || strings.Contains(curfile.Filename, "~") {
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Invalid characters in filename"}`))
return
}
// 1. Create the file object.
if len(basepath) == 0 {
basepath = "shuffle-files"
}
folderPath := fmt.Sprintf("%s/%s/%s", basepath, curfile.OrgId, curfile.WorkflowId)
// Try to make the full file location
err = os.MkdirAll(folderPath, os.ModePerm)
if err != nil {
log.Printf("[ERROR] Writing issue for file location creation: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Failed creating upload location"}`))
return
}
filename := curfile.Filename
fileId := uuid.NewV4().String()
downloadPath := fmt.Sprintf("%s/%s", folderPath, fileId)
timeNow := time.Now().Unix()
newFile := File{
Id: fileId,
CreatedAt: timeNow,
UpdatedAt: timeNow,
Description: "",
Status: "created",
Filename: filename,
OrgId: curfile.OrgId,
WorkflowId: curfile.WorkflowId,
DownloadPath: downloadPath,
}
err = setFile(ctx, newFile)
if err != nil {
log.Printf("[ERROR] Failed setting file: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "Failed setting file reference"}`))
return
} else {
log.Printf("[INFO] Created file %s", newFile.DownloadPath)
}
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"success": true, "id": "%s"}`, fileId)))
}
func getFile(ctx context.Context, id string) (*File, error) {
key := datastore.NameKey("Files", id, nil)
curFile := &File{}
if err := dbclient.Get(ctx, key, curFile); err != nil {
return &File{}, err
}
return curFile, nil
}
func setFile(ctx context.Context, file File) error {
// clear session_token and API_token for user
k := datastore.NameKey("Files", file.Id, nil)
if _, err := dbclient.Put(ctx, k, &file); err != nil {
log.Println(err)
return err
}
return nil
}
+18 -13
View File
@@ -2,35 +2,40 @@ module shuffle
go 1.13
//replace github.com/frikky/shuffle-shared => ../../../../git/shuffle-shared
//replace github.com/frikky/kin-openapi => ../../../../git/kin-openapi
require (
cloud.google.com/go v0.57.0
cloud.google.com/go/datastore v1.1.0
cloud.google.com/go v0.75.0
cloud.google.com/go/datastore v1.4.0
cloud.google.com/go/pubsub v1.3.1
cloud.google.com/go/storage v1.7.0
cloud.google.com/go/storage v1.12.0
github.com/Microsoft/go-winio v0.4.14 // indirect
github.com/basgys/goxml2json v1.1.0
github.com/frikky/kin-openapi v0.38.0
github.com/carlescere/scheduler v0.0.0-20170109141437-ee74d2f83d82
github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/docker/docker v1.13.1
github.com/docker/go-connections v0.4.0
github.com/docker/go-units v0.4.0 // indirect
github.com/getkin/kin-openapi v0.8.0
github.com/frikky/shuffle-shared v0.0.23
github.com/ghodss/yaml v1.0.0
github.com/go-git/go-billy/v5 v5.0.0
github.com/go-git/go-git/v5 v5.0.0
github.com/google/go-github/v28 v28.1.1
github.com/gorilla/handlers v1.4.2 // indirect
github.com/gorilla/mux v1.7.4
github.com/gorilla/mux v1.8.0
github.com/h2non/filetype v1.0.12
github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/satori/go.uuid v1.2.0
golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
google.golang.org/api v0.23.0
google.golang.org/appengine v1.6.6
google.golang.org/genproto v0.0.0-20200507105951-43844f6eee31
google.golang.org/grpc v1.29.1
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
golang.org/x/oauth2 v0.0.0-20210113160501-8b1d76fa0423
google.golang.org/api v0.36.0
google.golang.org/appengine v1.6.7
google.golang.org/genproto v0.0.0-20210113195801-ae06605f4595
google.golang.org/grpc v1.34.1
gopkg.in/src-d/go-git.v4 v4.13.1
gopkg.in/yaml.v2 v2.2.8
gopkg.in/yaml.v3 v3.0.0-20200506231410-2ff61e1afc86
gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
)
+171
View File
@@ -12,14 +12,24 @@ cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bP
cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
cloud.google.com/go v0.57.0 h1:EpMNVUorLiZIELdMZbCYX/ByTFCdoYopYAGxaGVz9ms=
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go v0.66.0/go.mod h1:dgqGAjKCDxyhGTtC9dAREQGUJpkceNm1yt590Qno0Ko=
cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=
cloud.google.com/go v0.75.0 h1:XgtDnVJRCPEUG21gjFiRPz4zI1Mjg16R+NYQjfmU4XY=
cloud.google.com/go v0.75.0/go.mod h1:VGuuCn7PG0dwsd5XPVm2Mm3wlh3EL55/79EKB6hlPTY=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
cloud.google.com/go/bigquery v1.6.0/go.mod h1:hyFDG0qSGdHNz8Q6nDN8rYIkld0q/+5uBZaelxiDLfE=
cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0 h1:/May9ojXjRkPBNVrq+oWLqmWCkr4OU5uRY29bu0mRyQ=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/datastore v1.4.0 h1:CFDJm15RpYXeEblQ0TMDUrYtqmBmbAWTy536nA8JIc8=
cloud.google.com/go/datastore v1.4.0/go.mod h1:d18825/a9bICdAIJy2EkHs9joU4RlIZ1t6l8WDdbdY0=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
@@ -30,6 +40,10 @@ cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0Zeo
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
cloud.google.com/go/storage v1.7.0 h1:DzdLPI8Em+DEk7IzA2a10ivq3mxIEASC9GeNJ6FFt5Q=
cloud.google.com/go/storage v1.7.0/go.mod h1:jGMIBwF+L/tL6WN/W5InNgYYu4HP0DvGB6rQ1mufWfs=
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
cloud.google.com/go/storage v1.12.0 h1:4y3gHptW1EHVtcPAVE0eBBlFuGqEejTTG3KdIE0lUX4=
cloud.google.com/go/storage v1.12.0/go.mod h1:fFLk2dp2oAhDz8QFKwqrjdJvxSp/W2g7nillojlL5Ho=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
@@ -49,6 +63,7 @@ github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5P
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -66,10 +81,27 @@ github.com/emirpasic/gods v1.12.0/go.mod h1:YfzfFFoVP/catgzJb4IKIqXjX78Ha8FMSDh3
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
github.com/frikky/kin-openapi v0.38.0 h1:V7ttwIJS8Vks4KL+mZVj1ZSqhIcQtgaG8akeqXEQgsE=
github.com/frikky/kin-openapi v0.38.0/go.mod h1:Fr28TtCHL4K0kIqtqui8HWxN1LG5uAh3z/tDfFyiA1s=
github.com/frikky/shuffle-shared v0.0.12 h1:+0EIfThmK47Po+LogPYZR4XjbS4Ds19WNMFu2YUSjhw=
github.com/frikky/shuffle-shared v0.0.12/go.mod h1:SEY432/xs4oBkOUnGwxiYKCZWZLRaheGInhAoW7N8ww=
github.com/frikky/shuffle-shared v0.0.15 h1:508ceeEHfPBMCC8/K4Zve3kwRQqiXNJSw6+BDoq9X4E=
github.com/frikky/shuffle-shared v0.0.15/go.mod h1:SEY432/xs4oBkOUnGwxiYKCZWZLRaheGInhAoW7N8ww=
github.com/frikky/shuffle-shared v0.0.20 h1:y6JlPnQDq//elICWvVfUfJyU9gH3fSpQmPy+agqZ5sA=
github.com/frikky/shuffle-shared v0.0.20/go.mod h1:H7SqOta/EAYnfYuWzwzYSh/oWfF0kgnuaJTQNKQBvoQ=
github.com/frikky/shuffle-shared v0.0.21 h1:xj/XPsXTa2rx41mm4nUc7+2K9RGkq2/mpjSPtIfpjE4=
github.com/frikky/shuffle-shared v0.0.21/go.mod h1:H7SqOta/EAYnfYuWzwzYSh/oWfF0kgnuaJTQNKQBvoQ=
github.com/frikky/shuffle-shared v0.0.22 h1:TFMcJCNmOOSneMMWbg5dNzp2z6m0aZLROuL+bzVToRE=
github.com/frikky/shuffle-shared v0.0.22/go.mod h1:H7SqOta/EAYnfYuWzwzYSh/oWfF0kgnuaJTQNKQBvoQ=
github.com/frikky/shuffle-shared v0.0.23 h1:Pnlc2M6fHnFRLFd5K1iLTVv4/t4P04Ri1GJ5CMxzq0U=
github.com/frikky/shuffle-shared v0.0.23/go.mod h1:H7SqOta/EAYnfYuWzwzYSh/oWfF0kgnuaJTQNKQBvoQ=
github.com/getkin/kin-openapi v0.8.0 h1:a6TQjTqwkyscC4/hShJX7WhCVE+4bi9lzw61XHQW5hE=
github.com/getkin/kin-openapi v0.8.0/go.mod h1:zZQMFkVgRHCdhgb6ihCTIo9dyDZFvX0k/xAKqw1FhPw=
github.com/getkin/kin-openapi v0.52.0 h1:6WqsF5d6PfJ8AscdD+9Rtb2RP2iBWyC7V6GcjssWg7M=
github.com/getkin/kin-openapi v0.52.0/go.mod h1:fRpo2Nw4Czgy0QnrIesRrEXs5+15N1F9mGZLP/aIomE=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gliderlabs/ssh v0.2.2/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0=
@@ -85,6 +117,10 @@ github.com/go-git/go-git/v5 v5.0.0/go.mod h1:oYD8y9kWsGINPFJoLdaScGCN6dlKg23blmC
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-openapi/jsonpointer v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY=
github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
github.com/go-openapi/swag v0.19.5 h1:lTz6Ys4CmqqCQmZPBlbQENR1/GucA2bzYTE12Pw4tFY=
github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
@@ -96,6 +132,7 @@ github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFU
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
@@ -108,6 +145,10 @@ github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrU
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0 h1:oOuy+ugB+P/kBdUnG5QaMXSIyJ1q38wWSojYCb3z5VQ=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3 h1:JjCZWpVbqXDqFVmTfYWEVTMIYrL/NPdPSCHPJ0T/raM=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
@@ -115,19 +156,32 @@ github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMyw
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0 h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.4 h1:L8R9j+yAqZuZjsqh/z+F1NCffTKKLShY6zXTItVIZ8M=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4rEjNlfyDHW9dolSY=
github.com/google/go-github/v28 v28.1.1 h1:kORf5ekX5qwXO2mGzXXOjMe/g6ap8ahVe0sBEulhSxo=
github.com/google/go-github/v28 v28.1.1/go.mod h1:bsqJWQX05omyWVmc00nEUql9mhQyv38lDZ8kPZcQVoM=
github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200905233945-acf8798be1f7/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5 h1:sjZBwGj9Jlw33ImPtvFviGYvseOtDM7hkSKB7+Tv3SM=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
@@ -135,11 +189,14 @@ github.com/gorilla/handlers v1.4.2 h1:0QniY0USkHQ1RGCLfKxeNHK9bkDHGRYGNDFBCS+YAR
github.com/gorilla/handlers v1.4.2/go.mod h1:Qkdc/uu4tH4g6mTK6auzZ766c4CA0Ng8+o/OAirnOIQ=
github.com/gorilla/mux v1.7.4 h1:VuZ8uybHlWmqV03+zRzdwKL4tUnIp1MAQtp1mIFE1bc=
github.com/gorilla/mux v1.7.4/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI=
github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/h2non/filetype v1.0.12 h1:yHCsIe0y2cvbDARtJhGBTD2ecvqMSTvlIcph9En/Zao=
github.com/h2non/filetype v1.0.12/go.mod h1:319b3zT68BvV+WRj7cwy856M2ehB3HqNOt6sy1HndBY=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
@@ -155,11 +212,16 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.8/go.mod h1:O1sed60cT9XZ5uDucP5qwvh+TE3NnUj51EiZO/lmSfw=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e h1:hB2xlXdHp/pmPZq0y3QnmWAArdw9PqbmotexnWx/FU8=
github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/opencontainers/go-digest v1.0.0-rc1 h1:WzifXhOVOEOuFYOJAW6aQqW0TooG2iki3E3Ii+WN7gQ=
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pelletier/go-buffruneio v0.2.0/go.mod h1:JkE26KsDizTr40EUHkXVtNPvgGtbSNq5BcowyYOWdKo=
github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -181,15 +243,21 @@ github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/xanzy/ssh-agent v0.2.1 h1:TCbipTQL2JiiCprBWx9frJ2eJlCYT00NmctrHxVAr70=
github.com/xanzy/ssh-agent v0.2.1/go.mod h1:mLlQY/MoOhWBj+gOGMQkOeiEvkx+8pJSI+0Bx9h2kr4=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3 h1:8sGtKOrtQqkN1bp2AtX+misvLIlOmsEsNd+9NIcPEm8=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5 h1:dntmOdLpSpHlVqbW5Eay97DelsZHe+55D+xC6i0dDS0=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
golang.org/x/crypto v0.0.0-20190219172222-a4c6cb3142f2/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
@@ -199,6 +267,8 @@ golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8U
golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79 h1:IaQbIIB2X/Mp/DKctl6ROxz1KyMlKp4uyvL6+kQ7C88=
golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 h1:psW17arqaxU48Z5kZ0CQnkZWQJsqcURM6tKiBApRjXI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -222,6 +292,7 @@ golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRu
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b h1:Wh+f8QHJXR411sJR8/vRBTZ7YapZaRvUcLFFJhusH0k=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
@@ -230,6 +301,9 @@ golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzB
golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0 h1:KU7oHjnv3XNWfa5COkzUifxZmxp1TyI7ImMXqFxLwvQ=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -240,6 +314,7 @@ golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
@@ -250,12 +325,28 @@ golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLL
golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5 h1:WQ8q63x+f/zpC8Ac1s9wLElVoHhm32p6tudrU72n1QA=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200904194848-62affa334b73/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b h1:iFwSg7t5GZmB/Q5TjiEAsdoLDrdJRC1RiF2WhuV29Qw=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210113160501-8b1d76fa0423 h1:/hEknzWkMPCjTo7StMHRrBRa8YBbXuBWfck8680k3RE=
golang.org/x/oauth2 v0.0.0-20210113160501-8b1d76fa0423/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -264,6 +355,9 @@ golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a h1:WXEvlFVvvGxCJLG6REjsT03iWnKLEWinaScsxF2Vm2o=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9 h1:SQFwaSi55rU7vdNs9Yr0Z324VNlrF+0wMqRXT4St8ck=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -289,11 +383,25 @@ golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200409092240-59c9f1ba88fa/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e h1:hq86ru83GdWTlfQFZGO4nZJTU4Bs2wfHl8oFHRaXsfc=
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200828194041-157a740278f4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3 h1:kzM6+9dur93BcC2kVlYl34cHU+TYZLanmpSJHVMmL64=
golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4 h1:0YWbFKbhXG/wIiuHDSKpS0Iy7FSA+u45VtBMfQcFTTc=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -334,10 +442,25 @@ golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWc
golang.org/x/tools v0.0.0-20200409170454-77362c5149f0/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d h1:lzLdP95xJmMpwQ6LUHwrc5V7js93hTiY7gkznu0BgmY=
golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200828161849-5deb26317202/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
golang.org/x/tools v0.0.0-20200915173823-2db8f0ff891c/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20200918232735-d647fc253266/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210114065538-d78b04bdf963/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
@@ -353,6 +476,15 @@ google.golang.org/api v0.21.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/
google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.23.0 h1:YlvGEOq2NA2my8cZ/9V8BcEO9okD48FlJcdqN0xJL3s=
google.golang.org/api v0.23.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
google.golang.org/api v0.31.0/go.mod h1:CL+9IBCa2WWU6gRuBWaKqGWLFFwbEUXkfeMkHLQWYWo=
google.golang.org/api v0.32.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/api v0.36.0 h1:l2Nfbl2GPXdWorv+dT2XfinX2jOOw4zv1VhLstx+6rE=
google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
@@ -360,6 +492,8 @@ google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.6 h1:lMO5rYAqUxkmaj76jAkRUvt5JZgFymx/+Q5Mzfivuhc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
@@ -385,6 +519,22 @@ google.golang.org/genproto v0.0.0-20200409111301-baae70f3302d/go.mod h1:55QSHmfG
google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200507105951-43844f6eee31 h1:Bz1qTn2YRWV+9OKJtxHJiQKCiXIdf+kwuKXdt9cBxyU=
google.golang.org/genproto v0.0.0-20200507105951-43844f6eee31/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200831141814-d751682dd103/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200914193844-75d14daec038/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200921151605-7abf4a1a14d5/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210108203827-ffc7fda8c3d7/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210113195801-ae06605f4595 h1:x7nk+/4+SvuTDI4wnzQUlhvi+DTpyfncXBo3QWTFs7U=
google.golang.org/genproto v0.0.0-20210113195801-ae06605f4595/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
@@ -397,12 +547,26 @@ google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKa
google.golang.org/grpc v1.28.1/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
google.golang.org/grpc v1.29.1 h1:EC2SB8S04d2r73uptxphDSUG+kTKVgjRPF+N3xpxRB4=
google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.32.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
google.golang.org/grpc v1.34.1 h1:ugq+9++ZQPFzM2pKUMCIK8gj9M0pFyuUWO9Q8kwEDQw=
google.golang.org/grpc v1.34.1/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0 h1:qdOKuR/EIArgaWNjetjgTzgVTAZ+S/WXVrq9HW9zimw=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
google.golang.org/protobuf v1.25.0 h1:Ejskq+SyPohKW+1uil0JJMtmHCgJPJ/qWTxr8qp+R4c=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -419,8 +583,14 @@ gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20200506231410-2ff61e1afc86 h1:OfFoIUYv/me30yv7XlMy4F9RJw8DEm8WQ6QG1Ph4bH0=
gopkg.in/yaml.v3 v3.0.0-20200506231410-2ff61e1afc86/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
@@ -428,6 +598,7 @@ honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWh
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2020.1.3 h1:sXmLre5bzIR6ypkjXCDI3jHPssRhc8KD/Ome589sc3U=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
+813 -2657
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+1652 -2406
View File
File diff suppressed because it is too large Load Diff
+2
View File
@@ -0,0 +1,2 @@
#!/bin/sh
curl -XPOST http://localhost:5001/api/v1/get_docker_image -d '{"name":"frikky/shuffle:Testing_1.0.0"}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
+25 -4
View File
@@ -1,5 +1,26 @@
#!/bin/sh
curl -XPOST http://localhost:5001/api/v1/workflows/1d9d8ce2-566e-4c3f-8a37-5d6c7d2000b5/execute -d '{"execution_argument":""}' -H "Authorization: Bearer 144308d0-6aab-4d4f-8bb2-75189281ee26"
curl -XPOST http://localhost:5001/api/v1/workflows/1d9d8ce2-566e-4c3f-8a37-5d6c7d2000b5/execute -d '{"execution_argument":""}' -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IjYwZjQwNjBlNThkNzVmZDNmNzBiZWZmODhjNzk0YTc3NTMyN2FhMzEiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJodHRwczovL3NodWZmbGVyLmlvL2FwaS92MS93b3JrZmxvd3MvMWQ5ZDhjZTItNTY2ZS00YzNmLThhMzctNWQ2YzdkMjAwMGI1L2V4ZWN1dGUiLCJhenAiOiIxMDMwNzY3ODIwNjE0MjQ2MTg0MjIiLCJlbWFpbCI6InNjaGVkdWxlckBzaHVmZmxlLTI0MTUxNy5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJleHAiOjE1NjU1Mjc1NTEsImlhdCI6MTU2NTUyMzk1MSwiaXNzIjoiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tIiwic3ViIjoiMTAzMDc2NzgyMDYxNDI0NjE4NDIyIn0.r0EDq9fjhf_5CPTiltyfk_L3uYJp577Uy0yYPcCAl2nv50_z_oUtbWGBpQLL8gcj-NGd3g4E52Qur8k6hCMIQweLS6WAb1279vGffEoCNDfkWb3Oy-yJGP1kzwLvqFJqnHLkSWYXNWvSyWnEimW8Rryx_m1BXS5wcA8l4NIr83kS7fPZrTwjnwFSeGSThwk91DVARzapQb8r0GEgOUyHZ1aBXnV98mikzSUt-5xFKe9eMdD22YJAj0Ru-DxAxs5nOqghX4PMRysWjshjOMrlR1piPWxqAmewp8YKZDCQ5gXskpeAFBDoULT971Wsx_NCohnJsFqx1JfPS9ZYMTW2oQ"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XPOST http://localhost:5001/api/v1/workflows/425efd39-08e7-4390-9387-170c172775f7/execute -d '{"execution_argument":""}' -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
+16
View File
@@ -0,0 +1,16 @@
curl http://192.168.3.6:5001/api/v1/files/create -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -d '{"filename": "file.txt", "org_id": "b199646b-16d2-456d-9fd6-b9972e929466", "workflow_id": "global"}'
curl http://localhost:5001/api/v1/files/create -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -d '{"filename": "file.txt", "org_id": "b199646b-16d2-456d-9fd6-b9972e929466", "workflow_id": "global"}'
echo
curl http://localhost:5001/api/v1/files/e19cffe4-e2da-47e9-809e-904f5cb03687/upload -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4" -F 'shuffle_file=@files.sh'
curl http://localhost:5001/api/v1/files/1915981b-b897-4db1-8a2e-44bc34cead3b/content -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl http://localhost:5001/api/v1/files/e19cffe4-e2da-47e9-809e-904f5cb03687 -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
curl -XDELETE http://localhost:5001/api/v1/files/e19cffe4-e2da-47e9-809e-904f5cb03687 -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
#r.HandleFunc("/api/v1/files/{fileId}/content", handleGetFileContent).Methods("GET", "OPTIONS")
#r.HandleFunc("/api/v1/files/create", handleCreateFile).Methods("POST", "OPTIONS")
#r.HandleFunc("/api/v1/files/{fileId}/upload", handleUploadFile).Methods("POST", "OPTIONS")
#r.HandleFunc("/api/v1/files/{fileId}", handleGetFileMeta).Methods("GET", "OPTIONS")
#r.HandleFunc("/api/v1/files/{fileId}", handleDeleteFile).Methods("DELETE", "OPTIONS")
+1
View File
@@ -0,0 +1 @@
curl http://localhost:5001/api/v1/apps/run_hotload -H "Authorization: Bearer e08c6f22-9a55-4557-b008-04388cc51fb0"
+1
View File
@@ -0,0 +1 @@
curl -XGET http://192.168.3.6:5001/api/v1/files -H "Authorization: Bearer db0373c6-1083-4dec-a05d-3ba73f02ccd4"
+1 -1
View File
@@ -1,2 +1,2 @@
# Fails cus of unmarshal
curl -XPOST http://localhost:5001/api/v1/workflows/1d9d8ce2-566e-4c3f-8a37-5d6c7d2000b5/schedule -d '{"name": "hey", "frequency": "*/1 * * * *", "execution_argument": "{\"test\": \"hey\"}"}' -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IjYwZjQwNjBlNThkNzVmZDNmNzBiZWZmODhjNzk0YTc3NTMyN2FhMzEiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJodHRwczovL3NodWZmbGVyLmlvL2FwaS92MS93b3JrZmxvd3MvMWQ5ZDhjZTItNTY2ZS00YzNmLThhMzctNWQ2YzdkMjAwMGI1L2V4ZWN1dGUiLCJhenAiOiIxMDMwNzY3ODIwNjE0MjQ2MTg0MjIiLCJlbWFpbCI6InNjaGVkdWxlckBzaHVmZmxlLTI0MTUxNy5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJleHAiOjE1NjU1Mjc1NTEsImlhdCI6MTU2NTUyMzk1MSwiaXNzIjoiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tIiwic3ViIjoiMTAzMDc2NzgyMDYxNDI0NjE4NDIyIn0.r0EDq9fjhf_5CPTiltyfk_L3uYJp577Uy0yYPcCAl2nv50_z_oUtbWGBpQLL8gcj-NGd3g4E52Qur8k6hCMIQweLS6WAb1279vGffEoCNDfkWb3Oy-yJGP1kzwLvqFJqnHLkSWYXNWvSyWnEimW8Rryx_m1BXS5wcA8l4NIr83kS7fPZrTwjnwFSeGSThwk91DVARzapQb8r0GEgOUyHZ1aBXnV98mikzSUt-5xFKe9eMdD22YJAj0Ru-DxAxs5nOqghX4PMRysWjshjOMrlR1piPWxqAmewp8YKZDCQ5gXskpeAFBDoULT971Wsx_NCohnJsFqx1JfPS9ZYMTW2oQ"
curl -XPOST http://localhost:5001/api/v1/workflows/1d9d8ce2-566e-4c3f-8a37-5d6c7d2000b5/schedule -d '{"name": "hey", "frequency": "*/1 * * * *", "execution_argument": "{\"test\": \"hey\"}"}' -H "Authorization: Bearer WUT"
+13
View File
@@ -0,0 +1,13 @@
curl http://localhost:5001/api/v1/users/register -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" -d '{"username": "username1", "password": ""}'
echo
curl http://localhost:5001/api/v1/users -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e"
echo UPDATE
curl -XPUT http://localhost:5001/api/v1/users/updateuser -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" -d '{"user_id": "id", "role": "admin"}'
echo
curl -XDELETE http://localhost:5001/api/v1/users/userid -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e"
echo
curl -XPOST http://localhost:5001/api/v1/users/generateapikey -H "Authorization: Bearer 36a3eb38-0070-41c6-b20a-2a3e0941d10e" -d '{"user_id": "390efa79-73a3-454b-8b1d-38f56eec14ad"}'
+8 -5
View File
@@ -2,7 +2,7 @@ version: '3'
services:
frontend:
#build: ./frontend
image: ghcr.io/frikky/shuffle-frontend:0.8.3
image: ghcr.io/frikky/shuffle-frontend:0.8.71
container_name: shuffle-frontend
hostname: shuffle-frontend
ports:
@@ -17,7 +17,7 @@ services:
- backend
backend:
#build: ./backend
image: ghcr.io/frikky/shuffle-backend:0.8.3
image: ghcr.io/frikky/shuffle-backend:0.8.71
container_name: shuffle-backend
hostname: ${BACKEND_HOSTNAME}
# Here for debugging:
@@ -35,9 +35,11 @@ services:
- SHUFFLE_FILE_LOCATION=/shuffle-files
- ORG_ID=${ORG_ID}
- SHUFFLE_APP_DOWNLOAD_LOCATION=${SHUFFLE_APP_DOWNLOAD_LOCATION}
- SHUFFLE_DOWNLOAD_AUTH_BRANCH=${SHUFFLE_DOWNLOAD_AUTH_BRANCH}
- SHUFFLE_DEFAULT_USERNAME=${SHUFFLE_DEFAULT_USERNAME}
- SHUFFLE_DEFAULT_PASSWORD=${SHUFFLE_DEFAULT_PASSWORD}
- SHUFFLE_DEFAULT_APIKEY=${SHUFFLE_DEFAULT_APIKEY}
- SHUFFLE_APP_FORCE_UPDATE=${SHUFFLE_APP_FORCE_UPDATE}
- HTTP_PROXY=${SHUFFLE_HTTP_PROXY}
- HTTPS_PROXY=${SHUFFLE_HTTPS_PROXY}
restart: unless-stopped
@@ -45,7 +47,7 @@ services:
- database
orborus:
#build: ./functions/onprem/orborus
image: ghcr.io/frikky/shuffle-orborus:0.8.0
image: ghcr.io/frikky/shuffle-orborus:0.8.71
container_name: shuffle-orborus
hostname: shuffle-orborus
networks:
@@ -53,8 +55,8 @@ services:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- SHUFFLE_APP_SDK_VERSION=0.8.0
- SHUFFLE_WORKER_VERSION=0.8.0
- SHUFFLE_APP_SDK_VERSION=0.8.60
- SHUFFLE_WORKER_VERSION=0.8.71
- ORG_ID=${ORG_ID}
- ENVIRONMENT_NAME=${ENVIRONMENT_NAME}
- BASE_URL=http://${OUTER_HOSTNAME}:${BACKEND_PORT}
@@ -66,6 +68,7 @@ services:
- SHUFFLE_BASE_IMAGE_NAME=${SHUFFLE_BASE_IMAGE_NAME}
- SHUFFLE_BASE_IMAGE_REGISTRY=${SHUFFLE_BASE_IMAGE_REGISTRY}
- SHUFFLE_BASE_IMAGE_TAG_SUFFIX=${SHUFFLE_BASE_IMAGE_TAG_SUFFIX}
- CLEANUP=${SHUFFLE_CONTAINER_AUTO_CLEANUP}
restart: unless-stopped
database:
#build: ./backend/database
+5 -8
View File
@@ -1,5 +1,5 @@
# Build environment
FROM node as builder
FROM node:14 as builder
RUN mkdir /usr/src/app
@@ -8,7 +8,8 @@ ENV PATH /usr/src/app/node_modules/.bin:$PATH
COPY package.json /usr/src/app/package.json
RUN npm install --verbose
#RUN npm install --verbose
RUN yarn install
# copy only required files to not trigger rebuilding every time
COPY ./certs /usr/src/app/certs/
@@ -17,14 +18,10 @@ COPY ./src /usr/src/app/src/
COPY ./*.sh /usr/src/app/
COPY ./*.json /usr/src/app/
# There were issues with the webpack installer from package.json
RUN rm -rf /usr/src/app/node_modules/webpack
RUN npm install webpack@4.42.0
RUN npm run-script build
RUN yarn build
# Production environment
FROM nginx:latest
FROM nginx:1.19
RUN mkdir -p /usr/share/nginx/html/build
RUN mkdir -p /usr/share/nginx/html/css
+927 -116
View File
File diff suppressed because it is too large Load Diff
+9 -7
View File
@@ -1,13 +1,14 @@
{
"name": "shuffler",
"homepage": "https://shuffler.io",
"version": "0.6.0",
"version": "0.8.53",
"private": true,
"dependencies": {
"@material-ui/core": "^4.5.2",
"@material-ui/data-grid": "^4.0.0-alpha.22",
"@material-ui/icons": "^4.5.1",
"@material-ui/styles": "^4.5.2",
"@use-it/interval": "^0.1.3",
"@use-it/interval": "^1.0.0",
"babel-eslint": "^10.1.0",
"class-transformer": "^0.3.1",
"create-react-app": "^2.0.3",
@@ -32,15 +33,16 @@
"md5-file": "^4.0.0",
"mdbreact": "^4.21.1",
"moment": "~2.20.1",
"react": "^16.10.2",
"react": "^16.14.0",
"react-alert": "^5.5.0",
"react-alert-template-basic": "^1.0.0",
"react-avatar-editor": "^11.1.0",
"react-beforeunload": "^2.2.1",
"react-chartjs-2": "^2.8.0",
"react-chartjs-2": "^2.11.1",
"react-cookie": "^4.0.1",
"react-cytoscapejs": "^1.2.0",
"react-device-detect": "^1.9.10",
"react-dom": "^16.10.2",
"react-dom": "^16.14.0",
"react-draggable": "^3.3.2",
"react-dropzone": "^10.1.10",
"react-ga": "^2.7.0",
@@ -52,12 +54,12 @@
"react-powerhooks": "0.0.7",
"react-router": "^4.3.1",
"react-router-dom": "^4.3.1",
"react-scripts": "^3.4.1",
"react-scripts": "^4.0.1",
"reactstrap": "^7.1.0",
"shellwords": "^0.1.1",
"simplebar": "^4.2.3",
"styled-components": "^4.4.0",
"webpack": "^4.42.0",
"webpack": "4.44.2",
"websocket": "^1.0.30",
"yaml": "^1.7.2",
"yamljs": "^0.3.0",
+10 -32
View File
@@ -13,10 +13,9 @@ import EditWebhook from "./views/EditWebhook";
import AngularWorkflow from "./views/AngularWorkflow";
import Header from './components/Header';
import theme from './theme'
import Apps from './views/Apps';
import AppCreator from './views/AppCreator';
import Contact from './views/Contact';
import Oauth2 from './views/Oauth2';
import Dashboard from "./views/Dashboard";
import AdminSetup from "./views/AdminSetup";
@@ -28,10 +27,14 @@ import LandingPageNew from "./views/LandingpageNew";
import LoginPage from "./views/LoginPage";
import SettingsPage from "./views/SettingsPage";
import MyView from "./views/MyView";
import { createMuiTheme, MuiThemeProvider } from '@material-ui/core/styles';
import ScrollToTop from "./components/ScrollToTop";
import AlertTemplate from "./components/AlertTemplate";
import { positions, Provider } from "react-alert";
import {isMobile} from "react-device-detect";
// Production - backend proxy forwarding in nginx
var globalUrl = window.location.origin
@@ -42,38 +45,13 @@ if (window.location.protocol == "http:" && window.location.port === "3000") {
//globalUrl = "http://localhost:5002"
}
const theme = createMuiTheme({
palette: {
primary: {
main: "#f85a3e"
},
secondary: {
main: '#e8eaf6',
},
surfaceColor: "#27292d",
inputColor: "#383B40"
},
typography: {
useNextVariants: true
},
overrides: {
MuiMenu: {
list: {
backgroundColor: "#383B40",
},
},
},
});
// FIXME - set client side cookies
const App = (message, props) => {
const [userdata, setUserData] = useState({});
//const [homePage, ] = useState(true);
const [cookies, setCookie, removeCookie] = useCookies([]);
const [isLoggedIn, setIsLoggedIn] = useState(false);
const [dataset, setDataset] = useState(false);
const [isLoaded, setIsLoaded] = useState(false);
const [curpath, setCurpath] = useState(typeof window === 'undefined' || window.location === undefined ? "" : window.location.pathname)
useEffect(() => {
if (dataset === false) {
@@ -126,9 +104,8 @@ const App = (message, props) => {
<Route exact path="/home" render={props => <LandingPageNew isLoaded={isLoaded} {...props} />} />
</div> :
<div style={{ backgroundColor: "#1F2023", color: "rgba(255, 255, 255, 0.65)", minHeight: "100vh" }}>
<ScrollToTop setCurpath={setCurpath} />
<Header cookies={cookies} removeCookie={removeCookie} isLoaded={isLoaded} globalUrl={globalUrl} setIsLoggedIn={setIsLoggedIn} isLoggedIn={isLoggedIn} userdata={userdata} {...props} />
<Route exact path="/oauth2" render={props => <Oauth2 isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/contact" render={props => <Contact isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/login" render={props => <LoginPage isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
<Route exact path="/admin" render={props => <Admin userdata={userdata} isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
<Route exact path="/admin/:key" render={props => <Admin isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
@@ -142,12 +119,13 @@ const App = (message, props) => {
<Route exact path="/apps/new" render={props => <AppCreator isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} {...props} />} />
<Route exact path="/apps/edit/:appid" render={props => <AppCreator isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} {...props} />} />
<Route exact path="/schedules/:key" render={props => <EditSchedule globalUrl={globalUrl} {...props} />} />
<Route exact path="/workflows" render={props => <Workflows cookies={cookies} removeCookie={removeCookie} isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} cookies={cookies} {...props} />} />
<Route exact path="/workflows" render={props => <Workflows cookies={cookies} removeCookie={removeCookie} isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} cookies={cookies} userdata={userdata} {...props} />} />
<Route exact path="/workflows/:key" render={props => <AngularWorkflow userdata={userdata} globalUrl={globalUrl} isLoaded={isLoaded} isLoggedIn={isLoggedIn} {...props} />} />
<Route exact path="/docs/:key" render={props => <Docs isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/docs/:key" render={props => <Docs isMobile={isMobile} isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/docs" render={props => { window.location.pathname = "/docs/about" }} />
<Route exact path="/introduction" render={props => <Introduction isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/introduction/:key" render={props => <Introduction isLoaded={isLoaded} globalUrl={globalUrl} {...props} />} />
<Route exact path="/myview" render={props => <MyView cookies={cookies} removeCookie={removeCookie} isLoaded={isLoaded} isLoggedIn={isLoggedIn} globalUrl={globalUrl} cookies={cookies} userdata={userdata} {...props} />} />
<Route exact path="/" render={props => <LoginPage isLoggedIn={isLoggedIn} setIsLoggedIn={setIsLoggedIn} register={true} isLoaded={isLoaded} globalUrl={globalUrl} setCookie={setCookie} cookies={cookies} {...props} />} />
</div>
File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 14 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 431 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 16 KiB

+1
View File
@@ -18,6 +18,7 @@ const alertStyle = {
width: 400,
boxSizing: 'border-box',
zIndex: 100001,
overflow: "hidden",
}
const buttonStyle = {
@@ -0,0 +1,97 @@
import React, {useState} from 'react';
import {Typography, } from '@material-ui/core';
const Workflow = (props) => {
const { workflow, appAuthentication, apps } = props
const [requiredActions, setRequiredActions] = React.useState([])
const [firstLoad, setFirstLoad] = React.useState("")
// Rofl
if (workflow === undefined || workflow === null) {
return null
}
if (apps === undefined || apps === null) {
return null
}
if (appAuthentication === undefined || appAuthentication === null) {
return null
}
if (firstLoad.length === 0 || firstLoad !== workflow.id) {
setFirstLoad(workflow.id)
const newactions = []
for (var key in workflow.actions) {
var newaction = {
"large_image": "",
"app_name": "",
"app_version": "",
"must_activate": false,
"must_authenticate": false,
"action_ids": [],
}
const action = workflow.actions[key]
console.log(action)
const app = apps.find(app => app.name === action.app_name && app.app_version === action.app_version)
if (app === undefined || app === null) {
console.log("COULDNT FIND APP - SEARCH BACKEND")
newaction.app_name = action.app_name
newaction.app_version = action.app_version
} else {
newaction.app_name = app.name
newaction.app_version = app.app_version
console.log("APP: ", app)
if (action.authentication_id === "" && app.authentication.required === true) {
console.log("Requires auth!")
newaction.must_authenticate = true
newaction.action_ids.push(action.id)
}
//newaction.app_name = action.app_name
//newaction.app_name = action.app_version
}
if (action.errors !== undefined && action.errors !== null && action.errors.length > 0) {
console.log("Has errors!")
}
console.log("NEWACTION: ", newaction)
if (newaction.must_authenticate || newaction.must_activate) {
newactions.push(newaction)
}
}
console.log("ACTIONS: ", newactions)
setRequiredActions(newactions)
}
const AppSection = (props) => {
const {action} = props
return (
<div>
<Typography variant="body2">Name: {action.app_name}:{action.app_version}. </Typography>
</div>
)
}
console.log(requiredActions)
return (
<div>
<Typography variant="h6">Workflow: {workflow.id}</Typography>
{requiredActions.map((data, index) => {
return (
<AppSection key={index} action={data} />
)
})}
</div>
)
}
export default Workflow
+62 -89
View File
@@ -4,10 +4,13 @@ import {BrowserView, MobileView} from "react-device-detect";
import {Link} from 'react-router-dom';
import List from '@material-ui/core/List';
import Avatar from '@material-ui/core/Avatar';
import Menu from '@material-ui/core/Menu';
import ListItem from '@material-ui/core/ListItem';
import MenuItem from '@material-ui/core/MenuItem';
import Select from '@material-ui/core/Select';
import Button from '@material-ui/core/Button';
import IconButton from '@material-ui/core/IconButton';
import HomeIcon from '@material-ui/icons/Home';
import PolymerIcon from '@material-ui/icons/Polymer';
import AppsIcon from '@material-ui/icons/Apps';
@@ -27,6 +30,7 @@ const Header = props => {
const [LoginHoverColor, setLoginHoverColor] = useState(hoverOutColor);
const [DocsHoverColor, setDocsHoverColor] = useState(hoverOutColor);
const [HelpHoverColor, setHelpHoverColor] = useState(hoverOutColor);
const [anchorEl, setAnchorEl] = React.useState(null);
const hrefStyle = {
color: hoverOutColor,
@@ -102,10 +106,55 @@ const Header = props => {
setLoginHoverColor(hoverOutColor)
}
const handleClick = (event) => {
setAnchorEl(event.currentTarget);
};
const handleClose = () => {
setAnchorEl(null);
};
// Should be based on some path
const logoCheck = !homePage ? null : null
const avatarMenu =
<span>
<IconButton color="primary" style={{marginRight: 15, }} aria-controls="simple-menu" aria-haspopup="true" onClick={(event) => {
setAnchorEl(event.currentTarget);
}}>
<Avatar style={{height: 35, width: 35,}} alt="Your username here" src="" />
</IconButton>
<Menu
id="simple-menu"
anchorEl={anchorEl}
keepMounted
open={Boolean(anchorEl)}
onClose={() => {
handleClose()
}}
>
<MenuItem onClick={(event) => {
event.preventDefault()
handleClose()
}}>
<Link to="/settings" style={hrefStyle}>
Settings
</Link>
</MenuItem>
<MenuItem style={{color: "white"}} onClick={(event) => {
event.preventDefault()
handleClose()
handleClickLogout()
}}>
Logout
</MenuItem>
</Menu>
</span>
// Handle top bar or something
const logoCheck = !homePage ? null : null
const loginTextBrowser = !isLoggedIn ?
<div style={{display: "flex"}}>
<List style={{display: "flex", flexDirect: "row"}} component="nav">
@@ -182,76 +231,16 @@ const Header = props => {
</List>
</div>
<div style={{flex: "10", display: "flex", flexDirection: "row-reverse"}}>
<List style={{display: 'flex', flexDirection: 'row-reverse'}} component="nav">
<ListItem style={{flex: "1", textAlign: "center"}}>
<div onMouseOver={handleLoginHover} onMouseOut={handleLoginHoverOut} onClick={handleClickLogout} style={{color: LoginHoverColor, cursor: "pointer"}}>
Logout
</div>
</ListItem>
{logoCheck}
<ListItem style={{flex: "1", textAlign: "center"}}>
<Link to="/settings" style={hrefStyle}>
<Button
style={{}}
variant="outlined"
color="primary"> Settings</Button>
</Link>
</ListItem>
{/*
<ListItem>
<Link to="/contact" style={hrefStyle}>
<Button
style={{}}
variant="contained"
color="primary"
>
Contact
</Button>
</Link>
</ListItem>
*/}
{userdata === undefined || userdata.admin === undefined || userdata.admin === null || !userdata.admin ? null :
<ListItem>
<Link to="/admin" style={hrefStyle}>
<Button
style={{}}
variant="contained"
color="primary"
>
Admin
</Button>
</Link>
</ListItem>
}
{userdata === undefined || userdata.orgs === undefined || userdata.orgs === null || userdata.orgs.length <= 1 ? null :
<ListItem>
<Select
SelectDisplayProps={{
style: {
marginLeft: 10,
}
}}
value={userdata.selected_org}
fullWidth
style={{backgroundColor: theme.palette.surfaceColor, color: "white", height: "50px"}}
onChange={(e) => {
console.log("SET ORG TO ", e.target.value)
}}
>
{userdata.orgs.map(data => {
return (
<MenuItem key={data.id} style={{backgroundColor: theme.palette.inputColor, color: "white"}} value={data}>
{data.name}
</MenuItem>
)
})}
</Select>
</ListItem>
}
</List>
{avatarMenu}
{userdata === undefined || userdata.admin === undefined || userdata.admin === null || !userdata.admin ? null :
<Link to="/admin" style={hrefStyle}>
<Button color="primary" variant="contained" style={{marginRight: 15, marginTop: 12}}>
Admin
</Button>
</Link>
}
</div>
</div>
</div>
const loginTextMobile = !isLoggedIn ?
<div style={{display: "flex"}}>
@@ -305,29 +294,13 @@ const Header = props => {
</List>
</div>
<div style={{flex: "10", display: "flex", flexDirection: "row-reverse"}}>
<List style={{display: 'flex', flexDirection: 'row-reverse'}} component="nav">
<ListItem style={{flex: "1", textAlign: "center"}}>
<div onMouseOver={handleLoginHover} onMouseOut={handleLoginHoverOut} onClick={handleClickLogout} style={{color: LoginHoverColor, cursor: "pointer"}}>
Logout
</div>
</ListItem>
{logoCheck}
<ListItem style={{flex: "1", textAlign: "center"}}>
<Link to="/settings" style={hrefStyle}>
<Button
style={{}}
variant="contained"
color="primary"> Settings</Button>
</Link>
</ListItem>
<ListItem></ListItem>
</List>
{avatarMenu}
</div>
</div>
</div>
// <Divider style={{height: "1px", width: "100%", backgroundColor: "rgb(91, 96, 100)"}}/>
const loadedCheck =
<div style={{minHeight: 68}}>
<div style={{minHeight: 60}}>
<BrowserView>
{loginTextBrowser}
</BrowserView>
+166 -6
View File
@@ -3,9 +3,15 @@ import { makeStyles } from '@material-ui/styles';
import { useTheme } from '@material-ui/core/styles';
import Tooltip from '@material-ui/core/Tooltip';
import Grid from '@material-ui/core/Grid';
import Button from '@material-ui/core/Button';
import TextField from '@material-ui/core/TextField';
import Typography from '@material-ui/core/Typography';
import { useAlert } from "react-alert";
import IconButton from '@material-ui/core/IconButton';
import ExpandLessIcon from '@material-ui/icons/ExpandLess';
import ExpandMoreIcon from '@material-ui/icons/ExpandMore';
import SaveIcon from '@material-ui/icons/Save';
const useStyles = makeStyles({
notchedOutline: {
@@ -23,11 +29,17 @@ const OrgHeader = (props) => {
const classes = useStyles()
var upload = ""
const defaultBranch = "master"
const [orgName, setOrgName] = React.useState(selectedOrganization.name)
const [orgDescription, setOrgDescription] = React.useState(selectedOrganization.description)
const [appDownloadUrl, setAppDownloadUrl] = React.useState(selectedOrganization.defaults === undefined ? "https://github.com/frikky/shuffle-apps" : selectedOrganization.defaults.app_download_repo === undefined || selectedOrganization.defaults.app_download_repo.length === 0 ? "https://github.com/frikky/shuffle-apps" : selectedOrganization.defaults.app_download_repo)
const [appDownloadBranch, setAppDownloadBranch] = React.useState(selectedOrganization.defaults === undefined ? defaultBranch : selectedOrganization.defaults.app_download_branch === undefined || selectedOrganization.defaults.app_download_branch.length === 0 ? defaultBranch : selectedOrganization.defaults.app_download_branch)
const [workflowDownloadUrl, setWorkflowDownloadUrl] = React.useState(selectedOrganization.defaults === undefined ? "https://github.com/frikky/shuffle-apps" : selectedOrganization.defaults.workflow_download_repo === undefined || selectedOrganization.defaults.workflow_download_repo.length === 0 ? "https://github.com/frikky/shuffle-workflows" : selectedOrganization.defaults.workflow_download_repo)
const [workflowDownloadBranch, setWorkflowDownloadBranch] = React.useState(selectedOrganization.defaults === undefined ? defaultBranch : selectedOrganization.defaults.workflow_download_branch === undefined || selectedOrganization.defaults.workflow_download_branch.length === 0 ? defaultBranch : selectedOrganization.defaults.workflow_download_branch)
const [file, setFile] = React.useState("")
const [fileBase64, setFileBase64] = React.useState(selectedOrganization.image)
const [expanded, setExpanded] = React.useState(false)
if (file !== "") {
const img = document.getElementById('logo')
@@ -55,12 +67,13 @@ const OrgHeader = (props) => {
}
}
const handleEditOrg = (name, description, orgId, image) => {
const handleEditOrg = (name, description, orgId, image, defaults) => {
const data = {
"name": name,
"description": description,
"org_id": orgId,
"image": image,
"defaults": defaults,
}
const url = globalUrl + `/api/v1/orgs/${selectedOrganization.id}`;
@@ -102,9 +115,14 @@ const OrgHeader = (props) => {
style={{ width: 150, height: 55, flex: 1 }}
variant="contained"
color="primary"
onClick={() => handleEditOrg(orgName, orgDescription, selectedOrganization.id, selectedOrganization.image)}
onClick={() => handleEditOrg(orgName, orgDescription, selectedOrganization.id, selectedOrganization.image, {
"app_download_repo": appDownloadUrl,
"app_download_branch": appDownloadBranch,
"workflow_download_repo": workflowDownloadUrl,
"workflow_download_branch": workflowDownloadBranch,
})}
>
Save Changes
<SaveIcon />
</Button>
var imageData = file.length > 0 ? file : fileBase64
@@ -113,7 +131,7 @@ const OrgHeader = (props) => {
return (
<div>
<div style={{color: "white", flex: "1", display: "flex", flexDirection: "row"}}>
<Tooltip title="Click to edit the app's image - 174x174" placement="bottom">
<Tooltip title="Click to edit the organizations's image (174x174)" placement="bottom">
<div style={{flex: "1", margin: "10px 25px 10px 0px", border: imageData !== undefined && imageData.length > 0 ? null : "1px solid #f85a3e", cursor: "pointer", backgroundColor: imageData !== undefined && imageData.length > 0 ? null : theme.palette.inputColor, maxWidth: 174, maxHeight: 174}} onClick={() => {upload.click()}}>
<input hidden type="file" ref={(ref) => upload = ref} onChange={editHeaderImage} />
{imageInfo}
@@ -188,9 +206,151 @@ const OrgHeader = (props) => {
<div style={{margin: "auto", textalign: "center",}}>
{orgSaveButton}
</div>
</div>
</div>
</div>
</div>
</div>
<div style={{textAlign: "center",}}>
<IconButton style={{color: "white", marginTop: 10, }} onClick={() => {
setExpanded(!expanded)
}}>
{expanded ?
<ExpandLessIcon />
:
<ExpandMoreIcon />
}
</IconButton>
{expanded ?
<Grid container spacing={3} style={{textAlign: "left"}}>
<Grid item xs={6} style={{}}>
<span>
<Typography>
App Download URL
</Typography>
<TextField
required
style={{flex: "1", marginTop: "5px", marginRight: "15px", backgroundColor: theme.palette.inputColor}}
fullWidth={true}
type="name"
id="outlined-with-placeholder"
margin="normal"
variant="outlined"
placeholder="A description for the organization"
value={appDownloadUrl}
onChange={e => {
setAppDownloadUrl(e.target.value)
}}
InputProps={{
classes: {
notchedOutline: classes.notchedOutline,
},
style:{
color: "white",
},
}}
/>
</span>
</Grid>
<Grid item xs={6} style={{}}>
<span>
<Typography>
App Download Branch
</Typography>
<TextField
required
style={{flex: "1", marginTop: "5px", marginRight: "15px", backgroundColor: theme.palette.inputColor}}
fullWidth={true}
type="name"
id="outlined-with-placeholder"
margin="normal"
variant="outlined"
placeholder="A description for the organization"
value={appDownloadBranch}
onChange={e => {
setAppDownloadBranch(e.target.value)
}}
InputProps={{
classes: {
notchedOutline: classes.notchedOutline,
},
style:{
color: "white",
},
}}
/>
</span>
</Grid>
<Grid item xs={6} style={{}}>
<span>
<Typography>
Workflow Download URL
</Typography>
<TextField
required
style={{flex: "1", marginTop: "5px", marginRight: "15px", backgroundColor: theme.palette.inputColor}}
fullWidth={true}
type="name"
id="outlined-with-placeholder"
margin="normal"
variant="outlined"
placeholder="A description for the organization"
value={workflowDownloadUrl}
onChange={e => {
setWorkflowDownloadUrl(e.target.value)
}}
InputProps={{
classes: {
notchedOutline: classes.notchedOutline,
},
style:{
color: "white",
},
}}
/>
</span>
</Grid>
<Grid item xs={6} style={{}}>
<span>
<Typography>
Workflow Download Branch
</Typography>
<TextField
required
style={{flex: "1", marginTop: "5px", marginRight: "15px", backgroundColor: theme.palette.inputColor}}
fullWidth={true}
type="name"
id="outlined-with-placeholder"
margin="normal"
variant="outlined"
placeholder="A description for the organization"
value={workflowDownloadBranch}
onChange={e => {
setWorkflowDownloadBranch(e.target.value)
}}
InputProps={{
classes: {
notchedOutline: classes.notchedOutline,
},
style:{
color: "white",
},
}}
/>
</span>
</Grid>
{/*
<span style={{textAlign: "center"}}>
{expanded ?
<ExpandLessIcon />
:
<ExpandMoreIcon />
}
</span>
*/}
</Grid>
:
null
}
</div>
</div>
)
}
+146
View File
@@ -0,0 +1,146 @@
import React, {useState, useEffect, useLayoutEffect} from 'react';
import * as cytoscape from 'cytoscape';
import CytoscapeComponent from 'react-cytoscapejs';
import cystyle from '../defaultCytoscapeStyle';
const surfaceColor = "#27292D"
const CytoscapeWrapper = (props) => {
const { globalUrl, inworkflow } = props;
const [elements, setElements] = useState([])
const [workflow, setWorkflow] = useState(inworkflow)
const [cy, setCy] = React.useState()
const bodyWidth = 200
const bodyHeight = 150
const setupGraph = () => {
const actions = workflow.actions.map(action => {
const node = {}
node.position = action.position
node.data = action
node.data._id = action["id"]
node.data.type = "ACTION"
node.isStartNode = action["id"] === workflow.start
var example = ""
if (action.example !== undefined && action.example !== null && action.example.length > 0) {
example = action.example
}
node.data.example = example
return node;
})
const triggers = workflow.triggers.map(trigger => {
const node = {}
node.position = trigger.position
node.data = trigger
node.data._id = trigger["id"]
node.data.type = "TRIGGER"
return node;
})
// FIXME - tmp branch update
var insertedNodes = [].concat(actions, triggers)
const edges = workflow.branches.map((branch, index) => {
//workflow.branches[index].conditions = [{
const edge = { };
var conditions = workflow.branches[index].conditions
if (conditions === undefined || conditions === null) {
conditions = []
}
var label = ""
if (conditions.length === 1) {
label = conditions.length+" condition"
} else if (conditions.length > 1) {
label = conditions.length+" conditions"
}
edge.data = {
id: branch.id,
_id: branch.id,
source: branch.source_id,
target: branch.destination_id,
label: label,
conditions: conditions,
hasErrors: branch.has_errors
};
// This is an attempt at prettier edges. The numbers are weird to work with.
/*
//http://manual.graphspace.org/projects/graphspace-python/en/latest/demos/edge-types.html
const sourcenode = actions.find(node => node.data._id === branch.source_id)
const destinationnode = actions.find(node => node.data._id === branch.destination_id)
if (sourcenode !== undefined && destinationnode !== undefined && branch.source_id !== branch.destination_id) {
//node.data._id = action["id"]
console.log("SOURCE: ", sourcenode.position)
console.log("DESTINATIONNODE: ", destinationnode.position)
var opposite = true
if (sourcenode.position.x > destinationnode.position.x) {
opposite = false
} else {
opposite = true
}
edge.style = {
'control-point-distance': opposite ? ["25%", "-75%"] : ["-10%", "90%"],
'control-point-weight': ['0.3', '0.7'],
}
}
*/
return edge;
})
setWorkflow(workflow)
// Verifies if a branch is valid and skips others
var newedges = []
for (var key in edges) {
var item = edges[key]
const sourcecheck = insertedNodes.find(data => data.data.id === item.data.source)
const destcheck = insertedNodes.find(data => data.data.id === item.data.target)
if (sourcecheck === undefined || destcheck === undefined) {
continue
}
newedges.push(item)
}
insertedNodes = insertedNodes.concat(newedges)
setElements(insertedNodes)
}
if (elements.length === 0) {
setupGraph()
}
return (
<CytoscapeComponent
elements={elements}
minZoom={0.35}
maxZoom={2.00}
style={{width: bodyWidth-15, height: bodyHeight-5, backgroundColor: surfaceColor}}
stylesheet={cystyle}
boxSelectionEnabled={true}
autounselectify={false}
showGrid={true}
cy={(incy) => {
// FIXME: There's something specific loading when
// you do the first hover of a node. Why is this different?
//console.log("CY: ", incy)
setCy(incy)
}}
/>
)
}
export default CytoscapeWrapper
+32
View File
@@ -0,0 +1,32 @@
import { useEffect } from 'react';
import { withRouter } from 'react-router-dom';
import ReactGA from 'react-ga';
function ScrollToTop({setCurpath, history }) {
useEffect(() => {
const unlisten = history.listen(() => {
window.scroll({
top: 0,
left: 0,
behavior: "smooth",
});
//ReactGA.event({
// category: "referral",
// action: "new_user_referral",
// label: "",
//})
ReactGA.pageview(window.location.pathname)
setCurpath(window.location.pathname)
});
return () => {
unlisten();
}
}, []);
return (null);
}
// https://stackoverflow.com/questions/36904185/react-router-scroll-to-top-on-every-transition
export default withRouter(ScrollToTop);
+37 -9
View File
@@ -6,7 +6,7 @@ const data = [{
'font-family': 'Segoe UI, Tahoma, Geneva, Verdana, sans-serif, sans-serif',
'font-weight': 'lighter',
'margin-right': '10px',
'font-size': '15px',
'font-size': '18px',
'width': '80px',
'height': '80px',
'color': 'white',
@@ -20,14 +20,15 @@ const data = [{
selector: 'edge',
css: {
'target-arrow-shape': 'triangle',
'target-arrow-color': 'yellow',
'target-arrow-color': 'grey',
'curve-style': 'unbundled-bezier',
'label': 'data(label)',
'text-margin-y': '-15px',
'width': '2px',
"color": "white",
"line-fill": "linear-gradient",
"line-gradient-stop-colors": ["cyan", "yellow"],
"line-gradient-stop-positions": ["0.0", "100"],
"line-gradient-stop-colors": ["grey", "grey"],
},
},
{
@@ -38,6 +39,21 @@ const data = [{
'border-color': '#81c784',
'background-width': '100%',
'background-height': '100%',
'border-radius': '5px',
},
},
{
selector: `node[app_name="Shuffle Tools"]`,
css: {
'width': '30px',
'height': '30px',
},
},
{
selector: `node[app_name="Testing"]`,
css: {
'width': '30px',
'height': '30px',
},
},
{
@@ -74,6 +90,8 @@ const data = [{
'shape': 'octagon',
'border-color': 'orange',
'background-color': '#213243',
'background-width': '100%',
'background-height': '100%',
},
},
{
@@ -99,6 +117,8 @@ const data = [{
css: {
'shape': 'ellipse',
'border-color': '#80deea',
'width': '80px',
'height': '80px',
},
},
{
@@ -109,7 +129,7 @@ const data = [{
},
},
{
selector: 'node:selected',
selector: ':selected',
css: {
'background-color': '#77b0d0',
'border-color': '#77b0d0',
@@ -161,7 +181,7 @@ const data = [{
css: {
'background-color': '#ffef47',
'border-color': '#ffef47',
'border-width': '5px',
'border-width': '8px',
'transition-property': 'border-width',
'transition-duration': '0.25s',
},
@@ -181,9 +201,11 @@ const data = [{
css: {
'background-color': "#f85a3e",
'border-color': '#f85a3e',
'border-width': '5px',
'border-width': '12px',
'transition-property': 'border-width',
'transition-duration': '0.25s',
'font-size': '30px',
'label': 'data(label)',
},
},
{
@@ -209,10 +231,13 @@ const data = [{
selector: 'edge.success-highlight',
css: {
'width': '5px',
'target-arrow-color': '#399645',
'line-color': '#399645',
'target-arrow-color': '#41dcab',
'line-color': '#41dcab',
'transition-property': 'line-color, width',
'transition-duration': '0.5s',
"line-fill": "linear-gradient",
"line-gradient-stop-positions": ["0.0", "100"],
"line-gradient-stop-colors": ["#41dcab", "#41dcab"],
},
},
{
@@ -220,7 +245,10 @@ const data = [{
css: {
'target-arrow-color': '#991818',
'line-color': '#991818',
'line-style': 'dashed'
'line-style': 'dashed',
"line-fill": "linear-gradient",
"line-gradient-stop-positions": ["0.0", "100"],
"line-gradient-stop-colors": ["#991818", "#991818"],
},
},
{
File diff suppressed because it is too large Load Diff
+1 -6
View File
@@ -6,11 +6,6 @@ import TextField from '@material-ui/core/TextField';
import Button from '@material-ui/core/Button';
import Paper from '@material-ui/core/Paper';
const hrefStyle = {
color: "white",
textDecoration: "none"
}
const bodyDivStyle = {
margin: "auto",
marginTop: "100px",
@@ -35,7 +30,7 @@ const useStyles = makeStyles({
});
const AdminAccount = props => {
const { globalUrl, isLoaded, isLoggedIn, setIsLoggedIn, setCookie, } = props;
const { globalUrl, isLoaded, isLoggedIn, } = props;
const [username, setUsername] = useState("");
const [password, setPassword] = useState("");
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large Load Diff
+222 -125
View File
@@ -2,50 +2,32 @@ import React, { useEffect } from 'react';
import { useInterval } from 'react-powerhooks';
import AppsIcon from '@material-ui/icons/Apps';
import Grid from '@material-ui/core/Grid';
import Select from '@material-ui/core/Select';
import Paper from '@material-ui/core/Paper';
import Divider from '@material-ui/core/Divider';
import ButtonBase from '@material-ui/core/ButtonBase';
import Button from '@material-ui/core/Button';
import TextField from '@material-ui/core/TextField';
import FormControl from '@material-ui/core/FormControl';
import MenuItem from '@material-ui/core/MenuItem';
import Tooltip from '@material-ui/core/Tooltip';
import FormControlLabel from '@material-ui/core/FormControlLabel';
import Switch from '@material-ui/core/Switch';
import Input from '@material-ui/core/Input';
import {IconButton, Typography, Grid, Select, Paper, Divider, ButtonBase, Button, TextField, FormControl, MenuItem, Tooltip, FormControlLabel, Switch, Input, Breadcrumbs, Chip, Dialog, DialogTitle, DialogActions, DialogContent, CircularProgress} from '@material-ui/core';
import {OpenInNew as OpenInNewIcon,Apps as AppsIcon, Cached as CachedIcon, Publish as PublishIcon, CloudDownload as CloudDownloadIcon, Edit as EditIcon, Delete as DeleteIcon} from '@material-ui/icons';
import { useTheme } from '@material-ui/core/styles';
import YAML from 'yaml'
import {Link} from 'react-router-dom';
import Breadcrumbs from '@material-ui/core/Breadcrumbs';
import ReactJson from 'react-json-view'
import Chip from '@material-ui/core/Chip';
import CachedIcon from '@material-ui/icons/Cached';
import CloudDownloadIcon from '@material-ui/icons/CloudDownload';
import PublishIcon from '@material-ui/icons/Publish';
import CloudDownload from '@material-ui/icons/CloudDownload';
import EditIcon from '@material-ui/icons/Edit';
import DeleteIcon from '@material-ui/icons/Delete';
import { useAlert } from "react-alert";
import Dialog from '@material-ui/core/Dialog';
import DialogTitle from '@material-ui/core/DialogTitle';
import DialogActions from '@material-ui/core/DialogActions';
import DialogContent from '@material-ui/core/DialogContent';
import CircularProgress from '@material-ui/core/CircularProgress';
import Dropzone from '../components/Dropzone';
const surfaceColor = "#27292D"
const inputColor = "#383B40"
const chipStyle = {
backgroundColor: "#3d3f43", height: 30, marginRight: 5, paddingLeft: 5, paddingRight: 5, height: 28, cursor: "pointer", borderColor: "#3d3f43", color: "white",
}
// Parses JSON data into keys that can be used everywhere :)
export const GetParsedPaths = (inputdata, basekey) => {
const splitkey = " > "
var parsedValues = []
if (inputdata === undefined || inputdata === null) {
return parsedValues
}
if (typeof(inputdata) !== "object") {
return parsedValues
}
@@ -106,9 +88,10 @@ export const GetParsedPaths = (inputdata, basekey) => {
const Apps = (props) => {
const { globalUrl, isLoggedIn, isLoaded } = props;
const { globalUrl, isLoggedIn, isLoaded, userdata } = props;
//const [workflows, setWorkflows] = React.useState([]);
const theme = useTheme();
const baseRepository = "https://github.com/frikky/shuffle-apps"
const alert = useAlert()
const [selectedApp, setSelectedApp] = React.useState({});
@@ -134,10 +117,12 @@ const Apps = (props) => {
const [field2, setField2] = React.useState("")
const [cursearch, setCursearch] = React.useState("")
const [sharingConfiguration, setSharingConfiguration] = React.useState("you")
const [downloadBranch, setDownloadBranch] = React.useState("master")
const [isDropzone, setIsDropzone] = React.useState(false);
const upload = React.useRef(null);
const isCloud = window.location.host === "localhost:3002" || window.location.host === "shuffler.io" ? true : false
const borderRadius = 3
const { start, stop } = useInterval({
duration: 5000,
@@ -161,6 +146,10 @@ const Apps = (props) => {
})
function sortByKey(array, key) {
if (array === undefined || array === null) {
return array
}
return array.sort(function(a, b) {
var x = a[key];
var y = b[key];
@@ -188,7 +177,7 @@ const Apps = (props) => {
minHeight: 130,
maxHeight: 130,
minWidth: "100%",
maxWidth: "100%",
maxWidth: 612.5,
marginBottom: 5,
borderRadius: 5,
color: "white",
@@ -210,25 +199,57 @@ const Apps = (props) => {
setIsLoading(false)
if (response.status !== 200) {
console.log("Status not 200 for apps :O!")
if (isCloud) {
window.location.pathname = "/search"
}
}
return response.json()
})
.then((responseJson) => {
responseJson = sortByKey(responseJson, "large_image")
//console.log("Apps: ", responseJson)
//responseJson = sortByKey(responseJson, "large_image")
//responseJson = sortByKey(responseJson, "is_valid")
//setFilteredApps(responseJson.filter(app => !internalIds.includes(app.name) && !(!app.activated && app.generated)))
var privateapps = []
var valid = []
var invalid = []
for (var key in responseJson) {
const app = responseJson[key]
if (app.is_valid && !(!app.activated && app.generated)) {
privateapps.push(app)
} else if (app.private_id !== undefined && app.private_id.length > 0) {
valid.push(app)
} else {
invalid.push(app)
}
}
setApps(responseJson)
setFilteredApps(responseJson)
if (responseJson.length > 0) {
setSelectedApp(responseJson[0])
if (responseJson[0].actions !== null && responseJson[0].actions.length > 0) {
setSelectedAction(responseJson[0].actions[0])
//console.log(privateapps)
//console.log(valid)
//console.log(invalid)
//console.log(privateapps)
//privateapps.reverse()
privateapps.push(...valid)
privateapps.push(...invalid)
setApps(privateapps)
setFilteredApps(privateapps)
if (privateapps.length > 0) {
if (selectedApp.id === undefined || selectedApp.id === null) {
setSelectedApp(privateapps[0])
}
if (privateapps[0].actions !== null && privateapps[0].actions.length > 0) {
setSelectedAction(privateapps[0].actions[0])
} else {
setSelectedAction({})
}
}
runAppSearch("")
//runAppSearch("")
})
.catch(error => {
alert.error(error.toString())
@@ -311,16 +332,22 @@ const Apps = (props) => {
boxColor = "orange"
}
if (data.invalid) {
boxColor = "red"
}
//<div style={{backgroundColor: theme.palette.inputColor, height: 100, width: 100, borderRadius: 3, verticalAlign: "middle", textAlign: "center", display: "table-cell"}}>
// <div style={{width: "100px", height: "100px", border: "1px solid black", verticalAlign: "middle", textAlign: "center", display: "table-cell"}}>
var imageline = data.large_image.length === 0 ?
<img alt={data.title} style={{width: 100, height: 100}} />
<img alt={data.title} style={{borderRadius: borderRadius, width: 100, height: 100, backgroundColor: theme.palette.inputColor,}} />
:
<img alt={data.title} src={data.large_image} style={{width: 100, height: 100, maxWidth: "100%"}} />
<img alt={data.title} src={data.large_image} style={{borderRadius: borderRadius, maxWidth: 100, minWidth: 100, maxHeight: "100%", display: "block", margin: "0 auto"}} onLoad={(event) => {
//console.log("IMG LOADED!: ", event.target)
}} />
// FIXME - add label to apps, as this might be slow with A LOT of apps
var newAppname = data.name
newAppname = newAppname.replace("_", " ")
newAppname = newAppname.charAt(0).toUpperCase()+newAppname.substring(1)
newAppname = newAppname.replaceAll("_", " ")
var sharing = "public"
if (!data.sharing) {
sharing = "private"
@@ -345,6 +372,7 @@ const Apps = (props) => {
return (
<Paper square key={data.id} style={paperAppStyle} onClick={() => {
if (selectedApp.id !== data.id) {
data.name = newAppname
setSelectedApp(data)
console.log(data)
@@ -355,27 +383,30 @@ const Apps = (props) => {
}
if (data.sharing) {
setSharingConfiguration("everyone")
setSharingConfiguration(isCloud ? "public" : "everyone")
}
}
}}>
<Grid container style={{margin: 10, flex: "10"}}>
<ButtonBase>
<Grid container style={{margin: 10, flex: "10", maxHeight: 110, overflow: "hidden",}}>
<ButtonBase style={{backgroundColor: theme.palette.inputColor, border: 3}}>
{imageline}
</ButtonBase>
<div style={{marginLeft: "10px", marginTop: "5px", marginBottom: "5px", width: boxWidth, backgroundColor: boxColor}}>
</div>
<Grid container style={{margin: "0px 10px 10px 10px", flex: "1"}}>
<Grid style={{display: "flex", flexDirection: "column", width: "100%"}}>
<div style={{marginLeft: "10px", marginTop: "5px", marginBottom: "5px", width: boxWidth, backgroundColor: boxColor}}/>
<Grid container style={{margin: "0px 0px 10px 10px", flex: "1"}}>
<Grid style={{display: "flex", flexDirection: "column", }}>
<Grid item style={{flex: "1"}}>
<h3 style={{marginBottom: "0px"}}>{newAppname}</h3>
<Typography variant="body1" style={{marginBottom: "0px", marginTop: 5,}}>
{newAppname}
</Typography>
</Grid>
<div style={{display: "flex", flex: "1"}}>
<Grid item style={{flex: "1", justifyContent: "center", overflow: "hidden"}}>
{description}
<div style={{display: "flex", flex: "1", marginTop: 5,}}>
<Grid item style={{flex: 1, justifyContent: "center", overflow: "hidden", maxHeight: 43, overflow: "hidden",}}>
<Typography variant="body2" color="textSecondary">
{description}
</Typography>
</Grid>
</div>
<Grid item style={{flex: "1", justifyContent: "center", marginTop: 5}}>
<Grid item style={{flex: 1, justifyContent: "center", marginTop: 8}}>
{data.tags === null || data.tags === undefined ? null : data.tags.map((tag, index) => {
if (index >= 3) {
return null
@@ -384,7 +415,7 @@ const Apps = (props) => {
return (
<Chip
key={index}
style={{height: 25, marginRight: 5, cursor: "pointer",}}
style={chipStyle}
label={tag}
variant="outlined"
color="primary"
@@ -399,7 +430,7 @@ const Apps = (props) => {
{data.activated && data.private_id !== undefined && data.private_id.length > 0 && data.generated ?
<Grid container style={{margin: "10px 10px 10px 10px", flex: "1"}} onClick={() => {downloadApp(data)}}>
<Tooltip title={"Download OpenAPI"} style={{marginTop: "28px", width: "100%"}} aria-label={data.name}>
<CloudDownload />
<CloudDownloadIcon />
</Tooltip>
</Grid>
: null}
@@ -426,7 +457,7 @@ const Apps = (props) => {
// FIXME - add label to apps, as this might be slow with A LOT of apps
var newAppname = selectedApp.name
if (newAppname !== undefined && newAppname.length > 0) {
newAppname = newAppname.replace("_", " ")
newAppname = newAppname.replaceAll("_", " ")
newAppname = newAppname.charAt(0).toUpperCase()+newAppname.substring(1)
} else {
newAppname = ""
@@ -446,7 +477,7 @@ const Apps = (props) => {
color="primary"
style={{marginTop: 10, marginRight: 8}}
>
<CloudDownload />
<CloudDownloadIcon />
</Button>
</Tooltip>
: null
@@ -515,9 +546,9 @@ const Apps = (props) => {
: null
var imageline = selectedApp.large_image === undefined || selectedApp.large_image.length === 0 ?
<img alt={selectedApp.title} style={{width: 100, height: 100}} />
<img alt={selectedApp.title} style={{borderRadius: borderRadius, width: 100, height: 100, backgroundColor: theme.palette.inputColor,}} />
:
<img alt={selectedApp.title} src={selectedApp.large_image} style={{width: 100, height: 100, maxWidth: "100%"}} />
<img alt={selectedApp.title} src={selectedApp.large_image} style={{borderRadius: borderRadius, maxWidth: 100, height: "auto", backgroundColor: theme.palette.inputColor}} />
const GetAppExample = () => {
if (selectedAction.returns === undefined) {
@@ -578,22 +609,36 @@ const Apps = (props) => {
const userRoles = [
"you",
"everyone",
isCloud ? "public" : "everyone",
]
//fetch(globalUrl+"/api/v1/get_openapi/"+urlParams.get("id"),
var baseInfo = newAppname.length > 0 ?
<div>
<div style={{position: "relative"}}>
<div style={{display: "flex"}}>
<div style={{marginRight: 15, marginTop: 10}}>
{imageline}
</div>
<div style={{maxWidth: "75%", overflow: "hidden"}}>
<h2 style={{marginTop: 20, marginBottom: 0, }}>{newAppname}</h2>
<p style={{marginTop: 5, marginBottom: 0,}}>Version {selectedApp.app_version}</p>
<p style={{marginTop: 5, marginBottom: 0}}>{description}</p>
<div style={{maxWidth: "85%", overflow: "hidden"}}>
<Typography variant="h6" style={{marginBottom: 0, }}>
{newAppname}
</Typography>
<Typography variant="body1" color="textSecondary">
Version {selectedApp.app_version}
</Typography>
<Typography variant="body2" color="textSecondary" style={{marginTop: 5, marginBottom: 0, maxHeight: 150, overflowY: "auto", overflowX: "hidden",}}>
{description}
</Typography>
</div>
</div>
{isCloud ?
<a href={"https://shuffler.io/apps/"+selectedApp.id} style={{textDecoration: "none", color: "#f85a3e"}} target="_blank">
<IconButton style={{top: -10, right: 0, position: "absolute", color: "#f85a3e"}} >
<OpenInNewIcon style={{}} />
</IconButton>
</a>
: null}
{activateButton}
{(props.userdata !== undefined && (props.userdata.role === "admin" || props.userdata.id === selectedApp.owner) || !selectedApp.generated) ?
<div>
@@ -612,7 +657,7 @@ const Apps = (props) => {
return (
<Chip
key={index}
style={{height: 25, marginRight: 5, marginTop: 7, cursor: "pointer",}}
style={chipStyle}
variant="outlined"
label={tag}
color="primary"
@@ -624,17 +669,23 @@ const Apps = (props) => {
{props.userdata !== undefined && props.userdata.id === selectedApp.owner ?
<div style={{marginTop: 15}}>
{/*<p><b>ID:</b> {selectedApp.id}</p>*/}
<b style={{marginRight: 15}}>Sharing:</b>
<b style={{marginRight: 15}}>Sharing </b>
<Select
value={sharingConfiguration}
onChange={(event) => {
setSharingConfiguration(event.target.value)
alert.info("Changed sharing to "+event.target.value)
alert.info("Changing sharing to "+event.target.value)
updateAppField(selectedApp.id, "sharing", !selectedApp.sharing)
//setSelectedAction(event.target.value)
setSharingConfiguration(event.target.value)
if (event.target.value === "you") {
updateAppField(selectedApp.id, "sharing", false)
} else if (event.target.value === "everyone" || event.target.value === "public") {
updateAppField(selectedApp.id, "sharing", true)
} else {
console.log("Can't handle value for sharing: ", event.target.value)
}
}}
style={{width: 150, backgroundColor: inputColor, color: "white", height: 35, marginleft: 10,}}
style={{width: 150, backgroundColor: theme.palette.surfaceColor, backgroundColor: inputColor, color: "white", height: 35, marginleft: 10,}}
SelectDisplayProps={{
style: {
marginLeft: 10,
@@ -675,12 +726,7 @@ const Apps = (props) => {
>
{selectedApp.actions.map(data => {
var newActionname = data.label !== undefined && data.label.length > 0 ? data.label : data.name
// ROFL FIXME - loop
newActionname = newActionname.replace("_", " ")
newActionname = newActionname.replace("_", " ")
newActionname = newActionname.replace("_", " ")
newActionname = newActionname.replace("_", " ")
newActionname = newActionname.replaceAll("_", " ")
newActionname = newActionname.charAt(0).toUpperCase()+newActionname.substring(1)
return (
<MenuItem key={data.name} style={{backgroundColor: inputColor, color: "white"}} value={data}>
@@ -699,7 +745,7 @@ const Apps = (props) => {
{selectedAction.parameters !== undefined && selectedAction.parameters !== null ?
<div style={{marginTop: 15, marginBottom: 15}}>
<b>Arguments</b>
<b>Parameters</b>
{selectedAction.parameters.map(data => {
var itemColor = "#f85a3e"
if (!data.required) {
@@ -739,7 +785,9 @@ const Apps = (props) => {
&nbsp;- <a href="https://apis.guru/browse-apis/" style={{textDecoration: "none", color: "#f85a3e"}} target="_blank">OpenAPI directory</a>
&nbsp;- <a href="https://editor.swagger.io/" style={{textDecoration: "none", color: "#f85a3e"}} target="_blank">OpenAPI Validator</a>
<div/>
Apps interact with eachother in workflows. They are created with the app creator, using OpenAPI specification or manually in python. The links above are references to OpenAPI tools and other app repositories. There's thousands of them.
<Typography variant="body2" color="textSecondary">
Apps interact with eachother in workflows. They are created with the app creator, using OpenAPI specification or manually in python. The links above are references to OpenAPI tools and other app repositories. There's thousands of them.
</Typography>
<div/>
<Divider style={{height: 1, backgroundColor: dividerColor, marginTop: 20, marginBottom: 20}} />
<div style={{}}>
@@ -757,7 +805,7 @@ const Apps = (props) => {
&nbsp;OR&nbsp;
<Link to="/apps/new" style={{marginLeft: 5, textDecoration: "none", color: "#f85a3e"}}>
<Button
variant="text"
variant="outlined"
component="label"
color="primary"
style={{}}
@@ -787,6 +835,7 @@ const Apps = (props) => {
var tmpapps = searchableApps.filter(data => data.name.toLowerCase().includes(searchfield) || data.description.toLowerCase().includes(searchfield))
newapps.push(...tmpapps)
console.log(newapps)
setFilteredApps(newapps)
//if ((newapps.length === 0 || searchBackend) && !appSearchLoading) {
@@ -802,12 +851,16 @@ const Apps = (props) => {
const reader = new FileReader();
reader.addEventListener('load', (e) => {
const content = e.target.result;
setOpenApiData(content);
setIsDropzone(isDropzone);
setOpenApiModal(true)
})
try {
reader.addEventListener('load', (e) => {
const content = e.target.result;
setOpenApiData(content);
setIsDropzone(isDropzone);
setOpenApiModal(true)
})
} catch (e) {
console.log("Error in dropzone: ", e)
}
reader.readAsText(files[0]);
};
@@ -829,29 +882,31 @@ const Apps = (props) => {
const appView = isLoggedIn ?
<Dropzone style={{maxWidth: window.innerWidth > 1366 ? 1366 : 1200, margin: "auto", padding: 20 }} onDrop={uploadFile}>
<div style={appViewStyle}>
<div style={{flex: 1}}>
<div style={{flex: 1, }}>
<Breadcrumbs aria-label="breadcrumb" separator="" style={{color: "white",}}>
<Link to="/apps" style={{textDecoration: "none", color: "inherit",}}>
<h2 style={{color: "rgba(255,255,255,0.5)"}}>
<Typography variant="h6" style={{color: "rgba(255,255,255,0.5)"}}>
<AppsIcon style={{marginRight: 10}} />
App upload
</h2>
</Typography>
</Link>
{selectedApp.activated && selectedApp.private_id !== undefined && selectedApp.private_id.length > 0 && selectedApp.generated ?
<Link to={`/apps/edit/${selectedApp.id}`} style={{textDecoration: "none", color: "inherit",}}>
<h2>
<Typography variant="h6">
{selectedApp.name}
</h2>
</Typography>
</Link>
: null}
</Breadcrumbs>
<div style={{marginTop: 15}} />
<UploadView/>
</div>
<Divider style={{marginBottom: 10, marginTop: 10, height: "100%", width: 1, backgroundColor: dividerColor}}/>
<div style={{flex: 1, marginLeft: 10, marginRight: 10}}>
<div style={{display: "flex", minHeight: 84.81}}>
<div style={{flex: 1}}>
<h2>Your apps ({apps.length+searchableApps.length})</h2>
<div style={{flex: 1, marginLeft: 10, marginRight: 10, }}>
<div style={{display: "flex",}}>
<div style={{flex: 1, marginBottom: 15, }}>
<Typography variant="h6">
Activated apps ({apps.length+searchableApps.length})
</Typography>
</div>
{isCloud ? null :
<span>
@@ -886,7 +941,7 @@ const Apps = (props) => {
}
</div>
<TextField
style={{backgroundColor: inputColor}}
style={{backgroundColor: inputColor, borderRadius: 5,}}
InputProps={{
style:{
color: "white",
@@ -894,6 +949,7 @@ const Apps = (props) => {
marginLeft: "5px",
maxWidth: "95%",
fontSize: "1em",
borderRadius: 5,
},
}}
fullWidth
@@ -907,7 +963,7 @@ const Apps = (props) => {
<div style={{marginTop: 15}}>
{apps.length > 0 ?
filteredApps.length > 0 ?
<div style={{height: "75vh", overflowY: "scroll"}}>
<div style={{height: "75vh", overflowY: "auto"}}>
{filteredApps.map(app => {
return (
appPaper(app)
@@ -923,9 +979,13 @@ const Apps = (props) => {
</div>
:
<Paper square style={uploadViewPaperStyle}>
<h4 style={{margin: 10, }}>
Try a broader search term, e.g. http, alert, ticket etc.
</h4>
<Typography style={{margin: 10, }}>
<span>
<a href={"https://shuffler.io/search"} style={{textDecoration: "none", color: "#f85a3e"}} target="_blank">
Click here
</a> to search ALL apps, not just your activated ones.
</span>
</Typography>
<div/>
{appSearchLoading ?
@@ -955,10 +1015,11 @@ const Apps = (props) => {
setValidation(true)
setIsLoading(true)
start()
//start()
const parsedData = {
"url": url,
"branch": downloadBranch || 'master'
}
if (field1.length > 0) {
@@ -986,20 +1047,25 @@ const Apps = (props) => {
if (response.status === 200) {
alert.success("Loaded existing apps!")
}
setIsLoading(false)
stop()
//stop()
setIsLoading(false)
setValidation(false)
return response.json()
})
.then((responseJson) => {
console.log("DATA: ", responseJson)
if (responseJson.reason !== undefined) {
alert.error("Failed loading: "+responseJson.reason)
}
console.log("DATA: ", responseJson)
if (responseJson.reason !== undefined) {
alert.error("Failed loading: "+responseJson.reason)
}
})
.catch(error => {
console.log("ERROR: ", error.toString())
alert.error(error.toString())
//stop()
setIsLoading(false)
setValidation(false)
})
}
@@ -1024,8 +1090,11 @@ const Apps = (props) => {
return response.json()
})
.then((responseJson) => {
if (responseJson.reason !== undefined && responseJson.reason.length > 0) {
alert.info("Hotloading: ", responseJson.reason)
if (responseJson.success === true) {
alert.info("Successfully finished hotload")
} else {
alert.error("Failed hotload: ", responseJson.reason)
//(responseJson.reason !== undefined && responseJson.reason.length > 0) {
}
})
.catch(error => {
@@ -1083,6 +1152,8 @@ const Apps = (props) => {
const data = {}
data[fieldname] = fieldvalue
console.log("DATA: ", data)
fetch(globalUrl+"/api/v1/apps/"+app_id, {
method: 'PATCH',
headers: {
@@ -1099,9 +1170,9 @@ const Apps = (props) => {
//console.log(responseJson)
//alert.info(responseJson)
if (responseJson.success) {
alert.info("Success")
alert.success("Successfully updated app configuration")
} else {
alert.error("Error updating app")
alert.error("Error updating app configuration")
}
})
.catch(error => {
@@ -1167,6 +1238,7 @@ const Apps = (props) => {
return
}
console.log("Validating response!")
validateOpenApi(responseJson)
})
.catch(error => {
@@ -1185,10 +1257,12 @@ const Apps = (props) => {
try {
return JSON.stringify(YAML.parse(apidata))
const parsed = YAML.parse(YAML.stringify(apidata))
//const parsed = YAML.parse(apidata))
return YAML.stringify(parsed)
} catch(error) {
console.log("YAML DECODE ERROR - TRY SOMETHING ELSE?: "+error)
setOpenApiError(error.toString())
setOpenApiError("Local error: "+ error.toString())
}
return ""
@@ -1197,19 +1271,23 @@ const Apps = (props) => {
// Sends the data to backend, which should return a version 3 of the same API
// If 200 - continue, otherwise, there's some issue somewhere
const validateOpenApi = (openApidata) => {
const newApidata = escapeApiData(openApidata)
var newApidata = escapeApiData(openApidata)
if (newApidata === "") {
// Used to return here
newApidata = openApidata
return
}
//console.log(newApidata)
setValidation(true)
fetch(globalUrl+"/api/v1/validate_openapi", {
method: 'POST',
method: 'POST',
headers: {
'Accept': 'application/json',
},
body: newApidata,
credentials: "include",
body: openApidata,
credentials: "include",
})
.then((response) => {
setValidation(false)
@@ -1302,7 +1380,7 @@ const Apps = (props) => {
style={{backgroundColor: inputColor}}
variant="outlined"
margin="normal"
defaultValue="https://github.com/frikky/shuffle-apps"
defaultValue={userdata.active_org.defaults.app_download_repo !== undefined && userdata.active_org.defaults.app_download_repo.length > 0 ? userdata.active_org.defaults.app_download_repo : "https://github.com/frikky/shuffle-apps"}
InputProps={{
style:{
color: "white",
@@ -1314,6 +1392,25 @@ const Apps = (props) => {
placeholder="https://github.com/frikky/shuffle-apps"
fullWidth
/>
<span style={{marginTop: 10}}>Branch (default value is "master"):</span>
<div style={{display: "flex"}}>
<TextField
style={{backgroundColor: inputColor}}
variant="outlined"
margin="normal"
defaultValue={userdata.active_org.defaults.app_download_branch !== undefined && userdata.active_org.defaults.app_download_branch.length > 0 ? userdata.active_org.defaults.app_download_branch : downloadBranch}
InputProps={{
style:{
color: "white",
height: "50px",
fontSize: "1em",
},
}}
onChange={e => setDownloadBranch(e.target.value)}
placeholder="master"
fullWidth
/>
</div>
<span style={{marginTop: 10}}>Authentication (optional - private repos etc):</span>
<div style={{display: "flex"}}>
+173 -140
View File
@@ -1,14 +1,12 @@
import React, {useState, useEffect} from 'react';
import React, {useState,} from 'react';
import Divider from '@material-ui/core/Divider';
import { useTheme } from '@material-ui/core/styles';
import ReactMarkdown from 'react-markdown';
import {BrowserView, MobileView} from "react-device-detect";
import Button from '@material-ui/core/Button';
import Menu from '@material-ui/core/Menu';
import MenuItem from '@material-ui/core/MenuItem';
import {Link} from 'react-router-dom';
import {Divider, Button, Menu, MenuItem, Typography, Paper, List} from '@material-ui/core';
const Body = {
maxWidth: '1000px',
minWidth: '768px',
@@ -20,26 +18,21 @@ const Body = {
};
const dividerColor = "rgb(225, 228, 232)"
const SideBar = {
maxWidth: 250,
flex: "1",
position: "fixed",
}
const hrefStyle = {
color: "rgba(255, 255, 255, 0.40)",
textDecoration: "none"
}
const Docs = (props) => {
const { isLoaded, globalUrl, inputColor } = props;
const { globalUrl, selectedDoc, serverside, isMobile, } = props;
const theme = useTheme();
const [data, setData] = useState("");
const [firstrequest, setFirstrequest] = useState(true);
const [list, setList] = useState([]);
const [listLoaded, setListLoaded] = useState(false);
const [, setListLoaded] = useState(false);
const [anchorEl, setAnchorEl] = React.useState(null);
const [baseUrl, setBaseUrl] = React.useState(serverside === true ? "" : window.location.href)
function handleClick(event) {
setAnchorEl(event.currentTarget);
@@ -49,77 +42,21 @@ const Docs = (props) => {
setAnchorEl(null);
}
useEffect(() => {
if (firstrequest) {
setFirstrequest(false)
fetchDocList()
fetchDocs(props.match.params.key)
return
}
const SidebarPaperStyle = {
backgroundColor: theme.palette.surfaceColor,
overflowX: "hidden",
position: "relative",
padding: 30,
paddingTop: 15,
borderRadius: 5,
}
// Continue this, and find the h2 with the data in it lol
if (window.location.hash.length > 0) {
var parent = document.getElementById("markdown_wrapper")
if (parent !== null) {
var elements = parent.getElementsByTagName('h2')
const name = window.location.hash.slice(1, window.location.hash.lenth).toLowerCase().split("%20").join(" ").split("_").join(" ").split("-").join(" ")
console.log(name)
var found = false
for (var key in elements) {
const element = elements[key]
if (element.innerHTML === undefined) {
continue
}
// Fix location..
if (element.innerHTML.toLowerCase() === name) {
element.scrollIntoView({behavior: "smooth"})
found = true
//element.scrollTo({
// top: element.offsetTop-100,
// behavior: "smooth"
//})
}
}
// H#
if (!found) {
var elements = parent.getElementsByTagName('h3')
console.log(name)
var found = false
for (var key in elements) {
const element = elements[key]
if (element.innerHTML === undefined) {
continue
}
// Fix location..
if (element.innerHTML.toLowerCase() === name) {
element.scrollIntoView({behavior: "smooth"})
found = true
//element.scrollTo({
// top: element.offsetTop-100,
// behavior: "smooth"
//})
}
}
}
}
//console.log(element)
//console.log("NAME: ", name)
//console.log(document.body.innerHTML)
// parent = document.getElementById(parent);
//var descendants = parent.getElementsByTagName(tagname);
// this.scrollDiv.current.scrollIntoView({ behavior: 'smooth' });
//$(".parent").find("h2:contains('Statistics')").parent();
}
})
const SideBar = {
maxWidth: 250,
flex: "1",
position: "fixed",
marginTop: 35,
}
const fetchDocList = () => {
fetch(globalUrl+"/api/v1/docs", {
@@ -143,16 +80,17 @@ const Docs = (props) => {
const fetchDocs = (docId) => {
fetch(globalUrl+"/api/v1/docs/"+docId, {
method: 'GET',
method: 'GET',
headers: {
'Content-Type': 'application/json',
'Accept': 'application/json',
},
})
})
.then((response) => response.json())
.then((responseJson) => {
.then((responseJson) => {
if (responseJson.success) {
setData(responseJson.reason)
document.title = "Shuffle "+docId+" documentation"
} else {
setData("# Error\nThis page doesn't exist.")
}
@@ -160,13 +98,99 @@ const Docs = (props) => {
.catch(error => {});
}
if (firstrequest) {
setFirstrequest(false)
if (selectedDoc !== undefined) {
setData(selectedDoc.reason)
setList(selectedDoc.list)
setListLoaded(true)
} else {
fetchDocList()
fetchDocs(props.match.params.key)
}
}
// Handles search-based changes that origin from outside this file
if (serverside !== true && window.location.href !== baseUrl) {
setBaseUrl(window.location.href)
fetchDocs(props.match.params.key)
}
const parseElementScroll = () => {
var parent = document.getElementById("markdown_wrapper_outer")
if (parent !== null) {
//console.log("IN PARENT")
var elements = parent.getElementsByTagName('h2')
const name = window.location.hash.slice(1, window.location.hash.lenth).toLowerCase().split("%20").join(" ").split("_").join(" ").split("-").join(" ")
//console.log(name)
var found = false
for (var key in elements) {
const element = elements[key]
if (element.innerHTML === undefined) {
continue
}
// Fix location..
if (element.innerHTML.toLowerCase() === name) {
element.scrollIntoView({behavior: "smooth"})
found = true
//element.scrollTo({
// top: element.offsetTop-100,
// behavior: "smooth"
//})
}
}
// H#
if (!found) {
elements = parent.getElementsByTagName('h3')
console.log(name)
found = false
for (key in elements) {
const element = elements[key]
if (element.innerHTML === undefined) {
continue
}
// Fix location..
if (element.innerHTML.toLowerCase() === name) {
element.scrollIntoView({behavior: "smooth"})
found = true
//element.scrollTo({
// top: element.offsetTop-100,
// behavior: "smooth"
//})
}
}
}
}
//console.log(element)
//console.log("NAME: ", name)
//console.log(document.body.innerHTML)
// parent = document.getElementById(parent);
//var descendants = parent.getElementsByTagName(tagname);
// this.scrollDiv.current.scrollIntoView({ behavior: 'smooth' });
//$(".parent").find("h2:contains('Statistics')").parent();
}
if (serverside !== true && window.location.hash.length > 0) {
parseElementScroll()
}
const markdownStyle = {
color: "rgba(255, 255, 255, 0.65)",
flex: "1",
maxWidth: 750,
maxWidth: isMobile ? "100%" : 750,
overflow: "hidden",
paddingBottom: 200,
marginLeft: 250,
marginLeft: isMobile ? 0 : 275,
}
function OuterLink(props) {
@@ -182,7 +206,7 @@ const Docs = (props) => {
function CodeHandler(props) {
return (
<pre style={{padding: 10, minWidth: "50%", maxWidth: "100%", backgroundColor: inputColor}}>
<pre style={{padding: 15, minWidth: "50%", maxWidth: "100%", backgroundColor: theme.palette.inputColor, overflowX: "auto", overflowY: "hidden",}}>
<code>
{props.value}
</code>
@@ -193,10 +217,10 @@ const Docs = (props) => {
function Heading(props) {
const element = React.createElement(`h${props.level}`, {style: {marginTop: 40}}, props.children)
return (
<span>
{props.level !== 1 ? <Divider style={{width: "90%", marginTop: 40, backgroundColor: inputColor}} /> : null}
<Typography>
{props.level !== 1 ? <Divider style={{width: "90%", marginTop: 40, backgroundColor: theme.palette.inputColor}} /> : null}
{element}
</span>
</Typography>
)
}
//React.createElement("p", {style: {color: "red", backgroundColor: "blue"}}, this.props.paragraph)
@@ -213,26 +237,28 @@ const Docs = (props) => {
const postDataBrowser =
<div style={Body}>
<div style={SideBar}>
<ul style={{listStyle: "none", paddingLeft: "0"}}>
{list.map((item, index) => {
const path = "/docs/"+item
const newname = item.charAt(0).toUpperCase()+item.substring(1).split("_").join(" ").split("-").join(" ")
return (
<li key={index} style={{marginTop: "10px"}}>
<Link style={hrefStyle} to={path} onClick={() => {fetchDocs(item)}}>
<h2>{newname}</h2>
</Link>
</li>
)
})}
</ul>
<Paper style={SidebarPaperStyle}>
<List style={{listStyle: "none", paddingLeft: "0", }}>
{list.map((item, index) => {
const path = "/docs/"+item
const newname = item.charAt(0).toUpperCase()+item.substring(1).split("_").join(" ").split("-").join(" ")
return (
<li key={index} style={{marginTop: 15,}}>
<Link key={index} style={hrefStyle} to={path} onClick={() => {fetchDocs(item)}}>
<Typography variant="h6"><b>{newname}</b></Typography>
</Link>
</li>
)
})}
</List>
</Paper>
</div>
<div id="markdown_wrapper" style={markdownStyle}>
<div id="markdown_wrapper_outer" style={markdownStyle}>
<ReactMarkdown
id="markdown_wrapper"
escapeHtml={false}
source={data}
renderers={{
renderers={{
link: OuterLink,
image: Img,
code: CodeHandler,
@@ -244,46 +270,56 @@ const Docs = (props) => {
const mobileStyle = {
color: "white",
marginLeft: "15px",
marginRight: "15px",
paddingBottom: "50px",
marginLeft: 25,
marginRight: 25,
paddingBottom: 50,
backgroundColor: "inherit",
display: "flex",
flexDirection: "column",
}
const postDataMobile =
<div style={mobileStyle}>
<Button aria-controls="simple-menu" aria-haspopup="true" variant="outlined" color="primary" onClick={handleClick}>
<div style={{color: "white"}}>
More items
</div>
</Button>
<Menu
id="simple-menu"
anchorEl={anchorEl}
keepMounted
open={Boolean(anchorEl)}
onClose={handleClose}
>
{list.map(item => {
const path = "/docs/"+item
const newname = item.charAt(0).toUpperCase()+item.substring(1).split("_").join(" ").split("-").join(" ")
return (
<MenuItem onClick={() => {window.location.pathname = path}}>{newname}</MenuItem>
)
})}
</Menu>
<div style={markdownStyle}>
<div>
<Button fullWidth aria-controls="simple-menu" aria-haspopup="true" variant="outlined" color="primary" onClick={handleClick}>
<div style={{color: "white"}}>
More docs
</div>
</Button>
<Menu
id="simple-menu"
anchorEl={anchorEl}
style={{}}
keepMounted
open={Boolean(anchorEl)}
onClose={handleClose}
>
{list.map((item, index) => {
const path = "/docs/"+item
const newname = item.charAt(0).toUpperCase()+item.substring(1).split("_").join(" ").split("-").join(" ")
return (
<MenuItem key={index} style={{color: "white",}} onClick={() => {window.location.pathname = path}}>{newname}</MenuItem>
)
})}
</Menu>
</div>
<div id="markdown_wrapper_outer" style={markdownStyle}>
<ReactMarkdown
id="markdown_wrapper"
escapeHtml={false}
source={data}
renderers={{link: OuterLink, image: Img}}
renderers={{
link: OuterLink,
image: Img,
code: CodeHandler,
heading: Heading,
}}
/>
</div>
<Divider style={{marginTop: "10px", marginBottom: "10px", backgroundColor: dividerColor}}/>
<Button aria-controls="simple-menu" aria-haspopup="true" variant="outlined" color="primary" onClick={handleClick}>
<Button fullWidth aria-controls="simple-menu" aria-haspopup="true" variant="outlined" color="primary" onClick={handleClick}>
<div style={{color: "white"}}>
More items
More docs
</div>
</Button>
@@ -297,7 +333,7 @@ const Docs = (props) => {
// {imageModal}
const loadedCheck = isLoaded && listLoaded ?
const loadedCheck =
<div>
<BrowserView>
{postDataBrowser}
@@ -306,9 +342,6 @@ const Docs = (props) => {
{postDataMobile}
</MobileView>
</div>
:
<div>
</div>
return (
<div>
+4 -3
View File
@@ -69,7 +69,7 @@ const LoginDialog = props => {
}),
)
.catch(error => {
setLoginInfo("Error in userdata: ", error)
setLoginInfo("Error logging in: ", error)
})
}
@@ -80,6 +80,7 @@ const LoginDialog = props => {
const onSubmit = (e) => {
e.preventDefault()
setLoginInfo("")
// FIXME - add some check here ROFL
// Just use this one?
@@ -114,7 +115,7 @@ const LoginDialog = props => {
}),
)
.catch(error => {
setLoginInfo("Error in userdata: " + error)
setLoginInfo("Error logging in: " + error)
});
} else {
url = baseurl + '/api/v1/users/register';
@@ -130,7 +131,7 @@ const LoginDialog = props => {
if (responseJson["success"] === false) {
setLoginInfo(responseJson["reason"])
} else {
setLoginInfo("Successful register :)")
setLoginInfo("Successful register!")
}
}),
)
File diff suppressed because it is too large Load Diff
+3 -7
View File
@@ -1,11 +1,7 @@
import React, {useState, useEffect} from 'react';
import Paper from '@material-ui/core/Paper';
import Button from '@material-ui/core/Button';
import Divider from '@material-ui/core/Divider';
import {Paper, Button, Divider, TextField} from '@material-ui/core';
import {Link} from 'react-router-dom';
import TextField from '@material-ui/core/TextField';
import { useAlert } from "react-alert";
import { useTheme } from '@material-ui/core/styles';
@@ -85,7 +81,7 @@ const Settings = (props) => {
const generateApikey = () => {
fetch(globalUrl+"/api/v1/generateapikey", {
method: 'GET',
method: 'GET',
headers: {
'Content-Type': 'application/json',
'Accept': 'application/json',
@@ -99,7 +95,7 @@ const Settings = (props) => {
return response.json()
})
.then((responseJson) => {
.then((responseJson) => {
setUserSettings(responseJson)
})
.catch(error => {
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,9 @@
GOOS=linux go build main.go
zip function.zip main
aws lambda update-function-code \
--function-name shuffler-forwarder \
--runtime go1.* \
--zip-file fileb://function.zip \
--handler main \
--role arn:aws:iam::123456789012:role/execution_role
+66
View File
@@ -0,0 +1,66 @@
package main
import (
"context"
//"encoding/json"
//"fmt"
"github.com/aws/aws-lambda-go/lambda"
"net/http"
)
type LambdaPayload struct {
RequestContext struct {
Elb struct {
TargetGroupArn string `json:"targetGroupArn"`
} `json:"elb"`
} `json:"requestContext"`
HTTPMethod string `json:"httpMethod"`
Path string `json:"path"`
Headers map[string]string `json:"headers"`
QueryStringParameters map[string]string `json:"queryStringParameters"`
Body string `json:"body"`
IsBase64Encoded bool `json:"isBase64Encoded"`
}
type LambdaResponse struct {
IsBase64Encoded bool `json:"isBase64Encoded"`
StatusCode int `json:"statusCode"`
StatusDescription string `json:"statusDescription"`
Headers struct {
SetCookie string `json:"Set-cookie"`
ContentType string `json:"Content-Type"`
} `json:"headers"`
Body string `json:"body"`
}
func lambda_handler(ctx context.Context, payload LambdaPayload) (LambdaResponse, error) {
response := &LambdaResponse{}
response.Headers.ContentType = "text/html"
response.StatusCode = http.StatusBadRequest
response.StatusDescription = http.StatusText(http.StatusBadRequest)
if payload.HTTPMethod == http.MethodGet && payload.Path == "/myfavoritecar" {
res := "TEST"
//car := &Car{}
//car.Model = "Corvette"
//car.Color = "Red"
//car.Year = 1999
//res, err := json.Marshal(car)
//if err != nil {
// fmt.Println(err)
// response.StatusCode = http.StatusInternalServerError
// response.StatusDescription = http.StatusText(http.StatusInternalServerError)
// return *response, err
//}
response.Headers.ContentType = "application/json"
response.Body = string(res)
response.StatusCode = http.StatusOK
response.StatusDescription = http.StatusText(http.StatusOK)
return *response, nil
} else {
return *response, nil
}
}
func main() {
lambda.Start(lambda_handler)
}
@@ -0,0 +1,35 @@
{
"name": "Shuffle",
"version": "1.0",
"author": "@frikkylikeme",
"url": "https://github.com/frikky/shuffle",
"license": "AGPL-V3",
"description": "Execute a workflow in Shuffle",
"dataTypeList": ["thehive:case", "thehive:alert"],
"command": "Shuffle/shuffle.py",
"baseConfig": "Shuffle",
"configurationItems": [
{
"name": "url",
"description": "The URL to your shuffle instance",
"type": "string",
"multi": false,
"required": true,
"defaultValue": "https://shuffler.io"
},
{
"name": "api_key",
"description": "The API key to your Shuffle user",
"type": "string",
"multi": false,
"required": true
},
{
"name": "workflow_id",
"description": "The ID of the workflow to execute",
"type": "string",
"multi": false,
"required": true
}
]
}
@@ -0,0 +1,28 @@
#!/usr/bin/env python
# encoding: utf-8
from cortexutils.responder import Responder
import requests
class Shuffle(Responder):
def __init__(self):
Responder.__init__(self)
self.api_key = self.get_param("config.api_key", "")
self.url = self.get_param("config.url", "")
self.workflow_id = self.get_param("config.workflow_id", "")
def run(self):
Responder.run(self)
parsed_url = "%s/api/v1/workflows/%s/execute" % (self.url, self.workflow_id)
headers = {
"Authorization": "Bearer %s" % self.api_key
}
requests.post(parsed_url, headers=headers)
self.report({'message': 'message sent'})
if __name__ == '__main__':
Shuffle().run()
+36
View File
@@ -0,0 +1,36 @@
#!/bin/sh
# Created by Shuffle, AS. <frikky@shuffler.io>.
WPYTHON_BIN="framework/python/bin/python3"
SCRIPT_PATH_NAME="$0"
DIR_NAME="$(cd $(dirname ${SCRIPT_PATH_NAME}); pwd -P)"
SCRIPT_NAME="$(basename ${SCRIPT_PATH_NAME})"
case ${DIR_NAME} in
*/active-response/bin | */wodles*)
if [ -z "${WAZUH_PATH}" ]; then
WAZUH_PATH="$(cd ${DIR_NAME}/../..; pwd)"
fi
PYTHON_SCRIPT="${DIR_NAME}/${SCRIPT_NAME}.py"
;;
*/bin)
if [ -z "${WAZUH_PATH}" ]; then
WAZUH_PATH="$(cd ${DIR_NAME}/..; pwd)"
fi
PYTHON_SCRIPT="${WAZUH_PATH}/framework/scripts/${SCRIPT_NAME}.py"
;;
*/integrations)
if [ -z "${WAZUH_PATH}" ]; then
WAZUH_PATH="$(cd ${DIR_NAME}/..; pwd)"
fi
PYTHON_SCRIPT="${DIR_NAME}/${SCRIPT_NAME}.py"
;;
esac
${WAZUH_PATH}/${WPYTHON_BIN} ${PYTHON_SCRIPT} "$@"
@@ -0,0 +1,177 @@
#!/usr/bin/env python
# Created by Shuffle, AS. <frikky@shuffler.io>.
# Based on the Slack integration using Webhooks
import json
import sys
import time
import os
try:
import requests
from requests.auth import HTTPBasicAuth
except Exception as e:
print("No module 'requests' found. Install: pip install requests")
sys.exit(1)
# ADD THIS TO ossec.conf configuration:
# <integration>
# <name>custom-shuffle</name>
# <hook_url>http://<IP>:3001/api/v1/hooks/<HOOK_ID></hook_url>
# <level>3</level>
# <alert_format>json</alert_format>
# </integration>
# Global vars
debug_enabled = False
pwd = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
json_alert = {}
now = time.strftime("%a %b %d %H:%M:%S %Z %Y")
# Set paths
log_file = '{0}/logs/integrations.log'.format(pwd)
def main(args):
debug("# Starting")
# Read args
alert_file_location = args[1]
webhook = args[3]
debug("# Webhook")
debug(webhook)
debug("# File location")
debug(alert_file_location)
# Load alert. Parse JSON object.
with open(alert_file_location) as alert_file:
json_alert = json.load(alert_file)
debug("# Processing alert")
debug(json_alert)
debug("# Generating message")
msg = generate_msg(json_alert)
if isinstance(msg, str):
if len(msg) == 0:
return
debug(msg)
debug("# Sending message")
send_msg(msg, webhook)
def debug(msg):
if debug_enabled:
msg = "{0}: {1}\n".format(now, msg)
print(msg)
f = open(log_file, "a")
f.write(msg)
f.close()
# Skips container kills to stop self-recursion
def filter_msg(alert):
# These are things that recursively happen because Shuffle starts Docker containers
# Docker integration rules: https://github.com/wazuh/wazuh-ruleset/blob/ae36745db1d3f312db0392f5925c2f2b0ec009a9/rules/0560-docker_integration_rules.xml
skip = ["87924", "87900", "87901", "87902", "87903", "87904", "86001", "86002", "86003", "87932", "80710", "87929", "87928",]
if alert["rule"]["id"] in skip:
return False
#try:
# if "docker" in alert["rule"]["description"].lower() and "
#msg['text'] = alert.get('full_log')
#except:
# pass
#msg['title'] = alert['rule']['description'] if 'description' in alert['rule'] else "N/A"
return True
def generate_msg(alert):
if not filter_msg(alert):
print("Skipping rule %s" % alert["rule"]["id"])
return ""
level = alert['rule']['level']
if (level <= 4):
color = "good"
elif (level >= 5 and level <= 7):
color = "warning"
else:
color = "danger"
msg = {}
msg['color'] = color
msg['pretext'] = "WAZUH Alert"
msg['title'] = alert['rule']['description'] if 'description' in alert['rule'] else "N/A"
msg['text'] = alert.get('full_log')
msg['rule_id'] = alert["rule"]["id"]
msg['timestamp'] = alert["timestamp"]
msg['id'] = alert['id']
msg["all_fields"] = alert
#msg['fields'] = []
# msg['fields'].append({
# "title": "Agent",
# "value": "({0}) - {1}".format(
# alert['agent']['id'],
# alert['agent']['name']
# ),
# })
#if 'agentless' in alert:
# msg['fields'].append({
# "title": "Agentless Host",
# "value": alert['agentless']['host'],
# })
#msg['fields'].append({"title": "Location", "value": alert['location']})
#msg['fields'].append({
# "title": "Rule ID",
# "value": "{0} _(Level {1})_".format(alert['rule']['id'], level),
#})
#attach = {'attachments': [msg]}
return json.dumps(msg)
def send_msg(msg, url):
headers = {'content-type': 'application/json', 'Accept-Charset': 'UTF-8'}
res = requests.post(url, data=msg, headers=headers)
debug(res)
if __name__ == "__main__":
try:
# Read arguments
bad_arguments = False
if len(sys.argv) >= 4:
msg = '{0} {1} {2} {3} {4}'.format(
now,
sys.argv[1],
sys.argv[2],
sys.argv[3],
sys.argv[4] if len(sys.argv) > 4 else '',
)
debug_enabled = (len(sys.argv) > 4 and sys.argv[4] == 'debug')
else:
msg = '{0} Wrong arguments'.format(now)
bad_arguments = True
# Logging the call
f = open(log_file, 'a')
f.write(msg + '\n')
f.close()
if bad_arguments:
debug("# Exiting: Bad arguments.")
sys.exit(1)
# Main function
main(sys.argv)
except Exception as e:
debug(str(e))
raise
+5
View File
@@ -0,0 +1,5 @@
<integration>
<name>custom-shuffle</name>
<hook_url>http://<IP>:3001/api/v1/hooks/webhook_<HOOK_ID></hook_url>
<alert_format>json</alert_format>
</integration>
+8 -2
View File
@@ -1,11 +1,17 @@
from golang as builder
FROM golang:1.16.0-buster as builder
RUN mkdir /app
WORKDIR /app
RUN go get github.com/docker/docker/api/types github.com/docker/docker/api/types/container github.com/docker/docker/client
COPY orborus.go /app/orborus.go
RUN go mod init orborus
RUN go get github.com/docker/docker/api/types && \
go get github.com/docker/docker/api/types/container && \
go get github.com/docker/docker/client && \
go get github.com/mackerelio/go-osstat/cpu && \
go get github.com/mackerelio/go-osstat/memory && \
go get github.com/satori/go.uuid && \
go get github.com/frikky/shuffle-shared
RUN go build
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o orborus .
+2 -2
View File
@@ -1,11 +1,11 @@
NAME=shuffle-orborus
VERSION=0.8.0
VERSION=0.8.71
echo "Running docker build with $NAME:$VERSION"
#docker rmi frikky/shuffle:$NAME --force
docker build . -t frikky/shuffle:$NAME -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t frikky/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
#docker push frikky/$NAME:$VERSION
#docker push frikky/shuffle:$NAME
# docker push docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION
docker push frikky/shuffle:$NAME
docker push ghcr.io/frikky/$NAME:$VERSION
+20
View File
@@ -0,0 +1,20 @@
module orborus
go 1.13
require (
github.com/Microsoft/go-winio v0.4.16 // indirect
github.com/containerd/containerd v1.4.3 // indirect
github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/docker/docker v20.10.1+incompatible
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.4.0 // indirect
github.com/frikky/shuffle-shared v0.0.23 // indirect
github.com/gogo/protobuf v1.3.1 // indirect
github.com/mackerelio/go-osstat v0.1.0
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.0.1 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/satori/go.uuid v1.2.0
github.com/sirupsen/logrus v1.7.0 // indirect
)
+502
View File
@@ -0,0 +1,502 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=
cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=
cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go v0.66.0/go.mod h1:dgqGAjKCDxyhGTtC9dAREQGUJpkceNm1yt590Qno0Ko=
cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=
cloud.google.com/go v0.75.0 h1:XgtDnVJRCPEUG21gjFiRPz4zI1Mjg16R+NYQjfmU4XY=
cloud.google.com/go v0.75.0/go.mod h1:VGuuCn7PG0dwsd5XPVm2Mm3wlh3EL55/79EKB6hlPTY=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/datastore v1.4.0 h1:CFDJm15RpYXeEblQ0TMDUrYtqmBmbAWTy536nA8JIc8=
cloud.google.com/go/datastore v1.4.0/go.mod h1:d18825/a9bICdAIJy2EkHs9joU4RlIZ1t6l8WDdbdY0=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=
cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
cloud.google.com/go/storage v1.12.0 h1:4y3gHptW1EHVtcPAVE0eBBlFuGqEejTTG3KdIE0lUX4=
cloud.google.com/go/storage v1.12.0/go.mod h1:fFLk2dp2oAhDz8QFKwqrjdJvxSp/W2g7nillojlL5Ho=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/Microsoft/go-winio v0.4.16 h1:FtSW/jqD+l4ba5iPBj9CODVtgfYAD8w2wS923g/cFDk=
github.com/Microsoft/go-winio v0.4.16/go.mod h1:XB6nPKklQyQ7GC9LdcBEcBl8PF76WugXOPRXwdLnMv0=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/containerd/containerd v1.4.3 h1:ijQT13JedHSHrQGWFcGEwzcNKrAGIiZ+jSD5QQG07SY=
github.com/containerd/containerd v1.4.3/go.mod h1:bC6axHOhabU15QhwfG7w5PipXdVtMXFTttgp+kVtyUA=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug=
github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v20.10.1+incompatible h1:u0HIBLwOJdemyBdTCkoBX34u3lb5KyBo0rQE3a5Yg+E=
github.com/docker/docker v20.10.1+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-units v0.4.0 h1:3uh0PgVws3nIA0Q+MwDC8yjEPf9zjRfZZWXZYDct3Tw=
github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/frikky/kin-openapi v0.38.0 h1:V7ttwIJS8Vks4KL+mZVj1ZSqhIcQtgaG8akeqXEQgsE=
github.com/frikky/kin-openapi v0.38.0/go.mod h1:Fr28TtCHL4K0kIqtqui8HWxN1LG5uAh3z/tDfFyiA1s=
github.com/frikky/shuffle-shared v0.0.12 h1:+0EIfThmK47Po+LogPYZR4XjbS4Ds19WNMFu2YUSjhw=
github.com/frikky/shuffle-shared v0.0.12/go.mod h1:SEY432/xs4oBkOUnGwxiYKCZWZLRaheGInhAoW7N8ww=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-openapi/jsonpointer v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY=
github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
github.com/go-openapi/swag v0.19.5 h1:lTz6Ys4CmqqCQmZPBlbQENR1/GucA2bzYTE12Pw4tFY=
github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
github.com/gogo/protobuf v1.3.1 h1:DqDEcV5aeaTmdFBePNpYsp3FlcVH/2ISVVM9Qf8PSls=
github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2 h1:+Z5KGCizgyZCbGh1KZqA0fcLLkwbsjIzS4aV2v7wJX0=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3 h1:JjCZWpVbqXDqFVmTfYWEVTMIYrL/NPdPSCHPJ0T/raM=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.4 h1:L8R9j+yAqZuZjsqh/z+F1NCffTKKLShY6zXTItVIZ8M=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200905233945-acf8798be1f7/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5 h1:sjZBwGj9Jlw33ImPtvFviGYvseOtDM7hkSKB7+Tv3SM=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1 h1:6QPYqodiu3GuPL+7mfx+NwDdp2eTkp9IfEUpgAwUN0o=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/mackerelio/go-osstat v0.1.0 h1:e57QHeHob8kKJ5FhcXGdzx5O6Ktuc5RHMDIkeqhgkFA=
github.com/mackerelio/go-osstat v0.1.0/go.mod h1:1K3NeYLhMHPvzUu+ePYXtoB58wkaRpxZsGClZBJyIFw=
github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e h1:hB2xlXdHp/pmPZq0y3QnmWAArdw9PqbmotexnWx/FU8=
github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVojFA6h/TRcI=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/satori/go.uuid v1.2.0 h1:0uYX9dsZ2yD7q2RtLRtPSdGDWzjeM3TbMJP9utgA0ww=
github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/sirupsen/logrus v1.7.0 h1:ShrD1U9pZB12TX0cVy0DtePoCH97K8EtX+mg7ZARUtM=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5 h1:dntmOdLpSpHlVqbW5Eay97DelsZHe+55D+xC6i0dDS0=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5 h1:2M3HP5CCK1Si9FQhwnzYhXdG6DXeebvUHFpre8QvbyI=
golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1 h1:Kvvh58BN8Y9/lBi7hTekvtMpm07eUZ0ck5pRHpsMWrY=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a h1:oWX7TPOiFAMXLq8o0ikBYfCJVlRHBcsciT5bXOrH628=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200904194848-62affa334b73/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b h1:iFwSg7t5GZmB/Q5TjiEAsdoLDrdJRC1RiF2WhuV29Qw=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210113160501-8b1d76fa0423 h1:/hEknzWkMPCjTo7StMHRrBRa8YBbXuBWfck8680k3RE=
golang.org/x/oauth2 v0.0.0-20210113160501-8b1d76fa0423/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190410235845-0ad05ae3009d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200828194041-157a740278f4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3 h1:kzM6+9dur93BcC2kVlYl34cHU+TYZLanmpSJHVMmL64=
golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4 h1:0YWbFKbhXG/wIiuHDSKpS0Iy7FSA+u45VtBMfQcFTTc=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200828161849-5deb26317202/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
golang.org/x/tools v0.0.0-20200915173823-2db8f0ff891c/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20200918232735-d647fc253266/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210114065538-d78b04bdf963 h1:K+NlvTLy0oONtRtkl1jRD9xIhnItbG2PiE7YOdjPb+k=
golang.org/x/tools v0.0.0-20210114065538-d78b04bdf963/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
google.golang.org/api v0.31.0/go.mod h1:CL+9IBCa2WWU6gRuBWaKqGWLFFwbEUXkfeMkHLQWYWo=
google.golang.org/api v0.32.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/api v0.36.0 h1:l2Nfbl2GPXdWorv+dT2XfinX2jOOw4zv1VhLstx+6rE=
google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013 h1:+kGHl1aib/qcwaRi1CbqBZ1rk19r85MNUf8HaBghugY=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200831141814-d751682dd103/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200914193844-75d14daec038/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200921151605-7abf4a1a14d5/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210108203827-ffc7fda8c3d7/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210113195801-ae06605f4595 h1:x7nk+/4+SvuTDI4wnzQUlhvi+DTpyfncXBo3QWTFs7U=
google.golang.org/genproto v0.0.0-20210113195801-ae06605f4595/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.32.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.34.0 h1:raiipEjMOIC/TO2AvyTxP25XFdLxNIBwzDh3FM3XztI=
google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
google.golang.org/grpc v1.34.1 h1:ugq+9++ZQPFzM2pKUMCIK8gj9M0pFyuUWO9Q8kwEDQw=
google.golang.org/grpc v1.34.1/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
google.golang.org/protobuf v1.25.0 h1:Ejskq+SyPohKW+1uil0JJMtmHCgJPJ/qWTxr8qp+R4c=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
+210 -49
View File
@@ -5,6 +5,8 @@ package main
*/
import (
"github.com/frikky/shuffle-shared"
"bytes"
"context"
"encoding/json"
@@ -21,17 +23,22 @@ import (
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
//"github.com/docker/docker/api/types/filters"
dockerclient "github.com/docker/docker/client"
"github.com/satori/go.uuid"
//network "github.com/docker/docker/api/types/network"
//natting "github.com/docker/go-connections/nat"
"github.com/mackerelio/go-osstat/cpu"
"github.com/mackerelio/go-osstat/memory"
)
// Starts jobs in bulk, so this could be increased
var sleepTime = 3
var maxConcurrency = 50
// Timeout if something rashes
var workerTimeoutEnv = os.Getenv("SHUFFLE_ORBORUS_EXECUTION_TIMEOUT")
var concurrencyEnv = os.Getenv("SHUFFLE_ORBORUS_EXECUTION_CONCURRENCY")
var appSdkVersion = os.Getenv("SHUFFLE_APP_SDK_VERSION")
var workerVersion = os.Getenv("SHUFFLE_WORKER_VERSION")
@@ -47,19 +54,8 @@ var baseUrl = os.Getenv("BASE_URL")
var environment = os.Getenv("ENVIRONMENT_NAME")
var dockerApiVersion = os.Getenv("DOCKER_API_VERSION")
var runningMode = strings.ToLower(os.Getenv("RUNNING_MODE"))
type ExecutionRequestWrapper struct {
Data []ExecutionRequest `json:"data"`
}
type ExecutionRequest struct {
ExecutionId string `json:"execution_id"`
ExecutionArgument string `json:"execution_argument"`
WorkflowId string `json:"workflow_id"`
Authorization string `json:"authorization"`
Status string `json:"status"`
Type string `json:"type"`
}
var cleanupEnv = strings.ToLower(os.Getenv("CLEANUP"))
var executionIds = []string{}
var dockercli *dockerclient.Client
var containerId string
@@ -99,17 +95,31 @@ func getThisContainerId() {
}
if fCol != "" {
cmd := fmt.Sprintf("cat /proc/self/cgroup | grep memory | tail -1 | cut -d/ -f%s", fCol)
cmd := fmt.Sprintf("cat /proc/self/cgroup | grep memory | tail -1 | cut -d/ -f%s | grep -o -E '[0-9A-z]{64}'", fCol)
out, err := exec.Command("bash", "-c", cmd).Output()
if err == nil {
containerId = strings.TrimSpace(string(out))
// cgroup error. Hardcoding this.
// https://github.com/moby/moby/issues/7015
//log.Printf("Checking if %s is in %s", ".scope", string(out))
if strings.Contains(string(out), ".scope") {
containerId = "shuffle-orborus"
//docker-76c537e9a4b7c7233011f5d70e6b7f2d600b6413ac58a96519b8dca7a3f7117a.scope
}
} else {
log.Printf("Failed getting container ID: %s", err)
if fCol == "0" {
containerId = "shuffle-orborus"
log.Printf("[WARNING] Failed getting container ID: %s", err)
}
}
}
log.Printf(`[INFO] Started with containerId "%s"`, containerId)
}
// Deploys the internal worker whenever something happens
// https://docs.docker.com/engine/api/sdk/examples/
func deployWorker(image string, identifier string, env []string) {
// Binds is the actual "-v" volume.
hostConfig := &container.HostConfig{
@@ -124,23 +134,28 @@ func deployWorker(image string, identifier string, env []string) {
// form container id and use it as network source if it's not empty
if containerId != "" {
log.Printf("[INFO] Found container ID %s", containerId)
//log.Printf("[INFO] Found container ID %s", containerId)
hostConfig.NetworkMode = container.NetworkMode(fmt.Sprintf("container:%s", containerId))
} else {
//log.Printf("[INFO] Empty self container id, continue without NetworkMode")
}
if cleanupEnv == "true" {
hostConfig.AutoRemove = true
}
config := &container.Config{
Image: image,
Env: env,
}
log.Printf("Identifier: %s", identifier)
//log.Printf("[INFO] Identifier: %s", identifier)
cont, err := dockercli.ContainerCreate(
context.Background(),
config,
hostConfig,
nil,
nil,
identifier,
)
@@ -148,12 +163,13 @@ func deployWorker(image string, identifier string, env []string) {
if strings.Contains(fmt.Sprintf("%s", err), "Conflict. The container name ") {
uuid := uuid.NewV4()
identifier = fmt.Sprintf("%s-%s", identifier, uuid)
log.Printf("2 - Identifier: %s", identifier)
log.Printf("[INFO] 2 - Identifier: %s", identifier)
cont, err = dockercli.ContainerCreate(
context.Background(),
config,
hostConfig,
nil,
nil,
identifier,
)
@@ -167,7 +183,8 @@ func deployWorker(image string, identifier string, env []string) {
}
}
err = dockercli.ContainerStart(context.Background(), cont.ID, types.ContainerStartOptions{})
containerStartOptions := types.ContainerStartOptions{}
err = dockercli.ContainerStart(context.Background(), cont.ID, containerStartOptions)
if err != nil {
log.Printf("[ERROR] Failed to start container in environment %s: %s", environment, err)
return
@@ -227,11 +244,11 @@ func initializeImages() {
ctx := context.Background()
if appSdkVersion == "" {
appSdkVersion = "0.8.0"
appSdkVersion = "0.8.60"
log.Printf("[WARNING] SHUFFLE_APP_SDK_VERSION not defined. Defaulting to %s", appSdkVersion)
}
if workerVersion == "" {
workerVersion = "0.8.0"
workerVersion = "0.8.70"
log.Printf("[WARNING] SHUFFLE_WORKER_VERSION not defined. Defaulting to %s", workerVersion)
}
@@ -248,9 +265,7 @@ func initializeImages() {
// check whether they are the same first
images := []string{
//fmt.Sprintf("%s/%s:app_sdk%s", baseimageregistry, baseimagename, baseimagetagsuffix),
//fmt.Sprintf("%s/%s:worker%s", baseimageregistry, baseimagename, baseimagetagsuffix),
fmt.Sprintf("frikky/shuffle:app_sdk"),
fmt.Sprintf("%s/%s/shuffle-app_sdk:%s", baseimageregistry, baseimagename, appSdkVersion),
fmt.Sprintf("%s/%s/shuffle-worker:%s", baseimageregistry, baseimagename, workerVersion),
// fmt.Sprintf("docker.io/%s:app_sdk", baseimagename),
@@ -275,6 +290,40 @@ func initializeImages() {
}
}
// Will be used for checking if there's enough to deploy based on a threshold
// E.g. having maximum CPU and maxmimum RAM
// Does this work containerized?
func getStats() {
fmt.Printf("\n")
memory, err := memory.Get()
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
return
}
before, err := cpu.Get()
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
return
}
time.Sleep(time.Duration(250) * time.Millisecond)
after, err := cpu.Get()
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
return
}
total := float64(after.Total - before.Total)
fmt.Printf("[INFO] memory total: %d bytes\n", memory.Total)
fmt.Printf("[INFO] memory used: %d bytes\n", memory.Used)
fmt.Printf("[INFO] cpu used : %f%%\n", float64(after.User-before.User)/total*100)
fmt.Printf("[INFO] cpu system: %f%%\n", float64(after.System-before.System)/total*100)
fmt.Printf("[INFO] cpu idle : %f%%\n", float64(after.Idle-before.Idle)/total*100)
fmt.Printf("\n")
}
// Initial loop etc
func main() {
log.Println("[INFO] Setting up execution environment")
@@ -302,7 +351,19 @@ func main() {
log.Printf("[INFO] Cleanup process running every %d seconds", workerTimeout)
}
go zombiecheck(workerTimeout)
if concurrencyEnv != "" {
//var concurrencyEnv = os.Getenv("SHUFFLE_ORBORUS_EXECUTION_CONCURRENCY")
tmpInt, err := strconv.Atoi(concurrencyEnv)
if err == nil {
maxConcurrency = tmpInt
log.Printf("[INFO] Max workflow execution concurrency set to %d", maxConcurrency)
} else {
log.Printf("[WARNING] Env SHUFFLE_ORBORUS_EXECUTION_CONCURRENCY must be a number, not %s. Defaulted to %d", workerTimeoutEnv, maxConcurrency)
}
}
ctx := context.Background()
go zombiecheck(ctx, workerTimeout)
log.Printf("[INFO] Running towards %s with Org %s", baseUrl, orgId)
httpProxy := os.Getenv("HTTP_PROXY")
@@ -337,6 +398,8 @@ func main() {
},
}
//getStats()
if (len(httpProxy) > 0 || len(httpsProxy) > 0) && baseUrl != "http://shuffle-backend:5001" {
client = &http.Client{}
} else {
@@ -367,13 +430,15 @@ func main() {
hasStarted := false
for {
//log.Printf("Prerequest")
//go getStats()
newresp, err := client.Do(req)
executionCount := getRunningWorkers(ctx, workerTimeout)
//log.Printf("Postrequest")
if err != nil {
log.Printf("[WARNING] Failed making request: %s", err)
zombiecounter += 1
if zombiecounter*sleepTime > workerTimeout {
go zombiecheck(workerTimeout)
go zombiecheck(ctx, workerTimeout)
zombiecounter = 0
}
time.Sleep(time.Duration(sleepTime) * time.Second)
@@ -394,21 +459,21 @@ func main() {
log.Printf("[ERROR] Failed reading body: %s", err)
zombiecounter += 1
if zombiecounter*sleepTime > workerTimeout {
go zombiecheck(workerTimeout)
go zombiecheck(ctx, workerTimeout)
zombiecounter = 0
}
time.Sleep(time.Duration(sleepTime) * time.Second)
continue
}
var executionRequests ExecutionRequestWrapper
var executionRequests shuffle.ExecutionRequestWrapper
err = json.Unmarshal(body, &executionRequests)
if err != nil {
log.Printf("[WARNING] Failed executionrequest in queue unmarshaling: %s", err)
sleepTime = 10
zombiecounter += 1
if zombiecounter*sleepTime > workerTimeout {
go zombiecheck(workerTimeout)
go zombiecheck(ctx, workerTimeout)
zombiecounter = 0
}
time.Sleep(time.Duration(sleepTime) * time.Second)
@@ -423,15 +488,31 @@ func main() {
if len(executionRequests.Data) == 0 {
zombiecounter += 1
if zombiecounter*sleepTime > workerTimeout {
go zombiecheck(workerTimeout)
go zombiecheck(ctx, workerTimeout)
zombiecounter = 0
}
time.Sleep(time.Duration(sleepTime) * time.Second)
continue
}
// Anything below here verifies concurrency virification
if executionCount >= maxConcurrency {
if zombiecounter*sleepTime > workerTimeout {
go zombiecheck(ctx, workerTimeout)
zombiecounter = 0
}
time.Sleep(time.Duration(sleepTime) * time.Second)
continue
}
allowed := maxConcurrency - executionCount
if len(executionRequests.Data) > allowed {
log.Printf("[WARNING] Throttle - Cutting down requests from %d to %d (MAX: %d, CUR: %d)", len(executionRequests.Data), allowed, maxConcurrency, executionCount)
executionRequests.Data = executionRequests.Data[0:allowed]
}
// New, abortable version. Should check executionid and remove everything else
var toBeRemoved ExecutionRequestWrapper
var toBeRemoved shuffle.ExecutionRequestWrapper
for _, execution := range executionRequests.Data {
if len(execution.ExecutionArgument) > 0 {
log.Printf("[INFO] Argument: %#v", execution.ExecutionArgument)
@@ -445,6 +526,24 @@ func main() {
if execution.Status == "ABORT" || execution.Status == "FAILED" {
log.Printf("[INFO] Executionstatus issue: ", execution.Status)
}
found := false
for _, executionId := range executionIds {
if execution.ExecutionId == executionId {
found = true
break
}
}
// Doesn't work because of USER INPUT
if found {
log.Printf("[INFO] Skipping duplicate %s", execution.ExecutionId)
continue
} else {
//log.Printf("[INFO] Adding to be ran %s", execution.ExecutionId)
executionIds = append(executionIds, execution.ExecutionId)
}
// Now, how do I execute this one?
// FIXME - if error, check the status of the running one. If it's bad, send data back.
containerName := fmt.Sprintf("worker-%s", execution.ExecutionId)
@@ -453,6 +552,7 @@ func main() {
fmt.Sprintf("EXECUTIONID=%s", execution.ExecutionId),
fmt.Sprintf("ENVIRONMENT_NAME=%s", environment),
fmt.Sprintf("BASE_URL=%s", baseUrl),
fmt.Sprintf("CLEANUP=%s", cleanupEnv),
}
if strings.ToLower(os.Getenv("SHUFFLE_PASS_WORKER_PROXY")) != "false" {
@@ -466,7 +566,7 @@ func main() {
go deployWorker(workerImage, containerName, env)
log.Printf("[INFO] %s is deployed and to be removed from queue.", execution.ExecutionId)
log.Printf("[INFO] ExecutionID %s was deployed and to be removed from queue.", execution.ExecutionId)
zombiecounter += 1
toBeRemoved.Data = append(toBeRemoved.Data, execution)
}
@@ -528,25 +628,22 @@ func main() {
}
}
// FIXME - add this to remove exited workers
// Should it check what happened to the execution? idk
func zombiecheck(workerTimeout int) error {
log.Println("[INFO] Looking for old containers")
ctx := context.Background()
// Is this ok to do with Docker? idk :)
func getRunningWorkers(ctx context.Context, workerTimeout int) int {
containers, err := dockercli.ContainerList(ctx, types.ContainerListOptions{
All: true,
})
//Filters: filters.Args{
// map[string][]string{"ancestor": {"<imagename>:<version>"}},
//},
if err != nil {
log.Printf("[ERROR] Failed creating Containerlist: %s", err)
return err
log.Printf("[ERROR] Error getting containers: %s", err)
return maxConcurrency
}
containerNames := map[string]string{}
stopContainers := []string{}
removeContainers := []string{}
currenttime := time.Now().Unix()
counter := 0
for _, container := range containers {
// Skip random containers. Only handle things related to Shuffle.
if !strings.Contains(container.Image, baseimagename) {
@@ -568,14 +665,76 @@ func zombiecheck(workerTimeout int) error {
for _, name := range container.Names {
// FIXME - add name_version_uid_uid regex check as well
if strings.HasPrefix(name, "/shuffle") {
if !strings.HasPrefix(name, "/worker") {
continue
}
log.Printf("[INFO] NAME: %s", name)
//log.Printf("Time: %d - %d", currenttime-container.Created, int64(workerTimeout))
if container.State == "running" && currenttime-container.Created < int64(workerTimeout) {
counter += 1
break
}
}
}
return counter
}
// FIXME - add this to remove exited workers
// Should it check what happened to the execution? idk
func zombiecheck(ctx context.Context, workerTimeout int) error {
executionIds = []string{}
log.Println("[INFO] Looking for old containers (zombies)")
containers, err := dockercli.ContainerList(ctx, types.ContainerListOptions{
All: true,
})
//log.Printf("Len: %d", len(containers))
if err != nil {
log.Printf("[ERROR] Failed creating Containerlist: %s", err)
return err
}
containerNames := map[string]string{}
stopContainers := []string{}
removeContainers := []string{}
log.Printf("[INFO] Baseimage: %s, Workertimeout: %d", baseimagename, int64(workerTimeout))
baseString := `/bin/sh -c 'python app.py --log-level DEBUG'`
for _, container := range containers {
// Skip random containers. Only handle things related to Shuffle.
if !strings.Contains(container.Image, baseimagename) && container.Command != baseString && container.Command != "./worker" {
shuffleFound := false
for _, item := range container.Labels {
if item == "shuffle" {
shuffleFound = true
break
}
}
// Check image name
if !shuffleFound {
log.Printf("Skipping: %s, %s", container.Labels, container.Image)
continue
}
//} else {
// log.Printf("NAME: %s", container.Image)
} else {
//log.Printf("Img: %s", container.Image)
//log.Printf("Names: %s", container.Names)
}
for _, name := range container.Names {
// FIXME - add name_version_uid_uid regex check as well
if strings.HasPrefix(name, "/shuffle") && !strings.HasPrefix(name, "/shuffle-subflow") {
continue
}
currenttime := time.Now().Unix()
//log.Printf("[INFO] (%s) NAME: %s. TIME: %d", container.State, name, currenttime-container.Created)
// Need to check time here too because a container can be removed the same instant as its created
currenttime := time.Now().Unix()
if container.State != "running" && currenttime-container.Created > int64(workerTimeout) {
removeContainers = append(removeContainers, container.ID)
containerNames[container.ID] = name
@@ -591,9 +750,10 @@ func zombiecheck(workerTimeout int) error {
}
// FIXME - add killing of apps with same execution ID too
log.Printf("[INFO] Should STOP %d containers.", len(stopContainers))
for _, containername := range stopContainers {
log.Printf("[INFO] Stopping and removing container %s", containerNames[containername])
go dockercli.ContainerStop(ctx, containername, nil)
dockercli.ContainerStop(ctx, containername, nil)
removeContainers = append(removeContainers, containername)
}
@@ -602,8 +762,9 @@ func zombiecheck(workerTimeout int) error {
Force: true,
}
log.Printf("[INFO] Should REMOVE %d containers.", len(removeContainers))
for _, containername := range removeContainers {
go dockercli.ContainerRemove(ctx, containername, removeOptions)
dockercli.ContainerRemove(ctx, containername, removeOptions)
}
return nil
+15 -6
View File
@@ -1,19 +1,28 @@
from golang as builder
FROM golang:1.16.0-buster as builder
WORKDIR /app
RUN go get github.com/docker/docker/api/types github.com/docker/docker/api/types/container github.com/docker/docker/client
RUN go get -u github.com/docker/docker/api/types
RUN go get -u github.com/docker/docker/api/types/container
RUN go get -u github.com/docker/docker/client
#RUN go env -w GO111MODULE=auto
COPY worker.go /app/worker.go
RUN go mod init worker
RUN go get github.com/docker/docker/api/types && \
go get github.com/docker/docker/api/types/container && \
go get github.com/docker/docker/client && \
go get github.com/gorilla/mux && \
go get github.com/patrickmn/go-cache && \
go get github.com/frikky/shuffle-shared && \
go get github.com/satori/go.uuid
RUN go build
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o worker .
## ALPINE IMAGE
FROM alpine:3.12
ENV SHUFFLE_BASE_IMAGE_REGISTRY=docker.io
ENV SHUFFLE_BASE_IMAGE_NAME=frikky/shuffle
ENV SHUFFLE_BASE_IMAGE_TAG_SUFFIX=0.6.0
ENV SHUFFLE_BASE_IMAGE_TAG_SUFFIX=0.8.70
RUN apk add --no-cache bash
COPY --from=builder /app/ /
+5 -3
View File
@@ -1,12 +1,14 @@
NAME=shuffle-worker
VERSION=0.8.0
VERSION=0.8.71
echo "Running docker build with $NAME:$VERSION"
#CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o worker.bin .
docker build . -t frikky/shuffle:$NAME -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
docker build . -t frikky/shuffle:$NAME -t frikky/shuffle:$NAME_$VERSION -t docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION -t ghcr.io/frikky/$NAME:$VERSION
# Push both for now..
#docker push frikky/$NAME:$VERSION
docker push frikky/shuffle:$NAME
#docker push frikky/shuffle:$NAME_$VERSION
#docker push docker.pkg.github.com/frikky/shuffle/$NAME:$VERSION
#docker tag frikky/shuffle:0.8.51 ghcr.io/frikky/shuffle-worker:0.8.5
#docker tag frikky/shuffle:$NAME ghcr.io/frikky/shuffle-worker:0.8.52
docker push ghcr.io/frikky/$NAME:$VERSION
+18
View File
@@ -0,0 +1,18 @@
module worker
go 1.15
require (
github.com/containerd/containerd v1.4.4 // indirect
github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/docker/docker v20.10.5+incompatible // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.4.0 // indirect
github.com/frikky/shuffle-shared v0.0.20 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/gorilla/mux v1.8.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.0.1 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/sirupsen/logrus v1.8.1 // indirect
)
File diff suppressed because it is too large Load Diff
-703
View File
@@ -1,703 +0,0 @@
package main
import (
"archive/zip"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"path/filepath"
"strings"
"archive/tar"
"cloud.google.com/go/storage"
"github.com/docker/docker/api/types"
"github.com/docker/docker/client"
"google.golang.org/api/cloudfunctions/v1"
"gopkg.in/yaml.v2"
)
var gceProject = "shuffler"
var bucketName = "shuffler.appspot.com"
type WorkflowAppActionParameter struct {
Description string `json:"description" datastore:"description"`
ID string `json:"id" datastore:"id"`
Name string `json:"name" datastore:"name"`
Example string `json:"example" datastore:"example"`
Value string `json:"value" datastore:"value"`
Multiline bool `json:"multiline" datastore:"multiline"`
ActionField string `json:"action_field" datastore:"action_field"`
Variant string `json:"variant", datastore:"variant"`
Required bool `json:"required" datastore:"required"`
Schema struct {
Type string `json:"type" datastore:"type"`
} `json:"schema"`
}
type Authentication struct {
Required bool `json:"required" datastore:"required" yaml:"required" `
Parameters []AuthenticationParams `json:"parameters" datastore:"parameters" yaml:"parameters"`
}
type AuthenticationParams struct {
Description string `json:"description" datastore:"description" yaml:"description"`
ID string `json:"id" datastore:"id" yaml:"id"`
Name string `json:"name" datastore:"name" yaml:"name"`
Example string `json:"example" datastore:"example" yaml:"example"`
Value string `json:"value" datastore:"value" yaml:"value"`
Multiline bool `json:"multiline" datastore:"multiline" yaml:"multiline"`
Required bool `json:"required" datastore:"required" yaml:"required"`
}
type WorkflowApp struct {
Name string `json:"name" yaml:"name" required:true datastore:"name"`
IsValid bool `json:"is_valid" yaml:"is_valid" required:true datastore:"is_valid"`
ID string `json:"id" yaml:"id" required:false datastore:"id"`
Link string `json:"link" yaml:"link" required:false datastore:"link"`
AppVersion string `json:"app_version" yaml:"app_version" required:true datastore:"app_version"`
Description string `json:"description" datastore:"description" required:false yaml:"description"`
Environment string `json:"environment" datastore:"environment" required:true yaml:"environment"`
Sharing bool `json:"sharing" datastore:"sharing" yaml:"sharing"`
SmallImage string `json:"small_image" datastore:"small_image" required:false yaml:"small_image"`
LargeImage string `json:"large_image" datastore:"large_image" yaml:"large_image" requred:false`
ContactInfo struct {
Name string `json:"name" datastore:"name" yaml:"name"`
Url string `json:"url" datastore:"url" yaml:"url"`
} `json:"contact_info" datastore:"contact_info" yaml:"contact_info" required:false`
Actions []WorkflowAppAction `json:"actions" yaml:"actions" required:true datastore:"actions"`
Authentication Authentication `json:"authentication" yaml:"authentication" required:false datastore:"authentication"`
}
type AuthenticationStore struct {
Key string `json:"key" datastore:"key"`
Value string `json:"value" datastore:"value"`
}
type WorkflowAppAction struct {
Description string `json:"description" datastore:"description"`
ID string `json:"id" datastore:"id"`
Name string `json:"name" datastore:"name"`
NodeType string `json:"node_type" datastore:"node_type"`
Environment string `json:"environment" datastore:"environment"`
Parameters []WorkflowAppActionParameter `json:"parameters" datastore: "parameters"`
Authentication []AuthenticationStore `json:"authentication" datastore:"authentication"`
Returns struct {
Description string `json:"description" datastore:"returns"`
ID string `json:"id" datastore:"id"`
Schema struct {
Type string `json:"type" datastore:"type"`
} `json:"schema" datastore:"schema"`
} `json:"returns" datastore:"returns"`
}
func getRunner(classname string) string {
return fmt.Sprintf(`
# Run the actual thing after we've checked params
def run(request):
action = request.get_json()
print(action)
print(type(action))
authorization_key = action.get("authorization")
current_execution_id = action.get("execution_id")
if action and "name" in action and "app_name" in action:
asyncio.run(%s.run(action), debug=True)
return f'Attempting to execute function {action["name"]} in app {action["app_name"]}'
else:
return f'Invalid action'
`, classname)
}
// Could use some kind of linting system too for this, but meh
func formatAppfile(filedata []byte) (string, []byte) {
lines := strings.Split(string(filedata), "\n")
newfile := []string{}
classname := ""
for _, line := range lines {
if strings.Contains(line, "walkoff_app_sdk") {
continue
}
// Remap logging. CBA this right now
// This issue also persists in onprem apps because of await thingies.. :(
// FIXME
if strings.Contains(line, "console_logger") && strings.Contains(line, "await") {
continue
//line = strings.Replace(line, "console_logger", "logger", -1)
//log.Println(line)
}
// Might not work with different import names
// Could be fucked up with spaces everywhere? Idk
if strings.Contains(line, "class") && strings.Contains(line, "(AppBase)") {
items := strings.Split(line, " ")
if len(items) > 0 && strings.Contains(items[1], "(AppBase)") {
classname = strings.Split(items[1], "(")[0]
} else {
log.Println("Something wrong :( (horrible programming right here)")
os.Exit(3)
}
}
if strings.Contains(line, "if __name__ ==") {
break
}
// asyncio.run(HelloWorld.run(), debug=True)
newfile = append(newfile, line)
}
filedata = []byte(strings.Join(newfile, "\n"))
return classname, filedata
}
// https://stackoverflow.com/questions/21060945/simple-way-to-copy-a-file-in-golang
func Copy(src, dst string) error {
in, err := os.Open(src)
if err != nil {
return err
}
defer in.Close()
out, err := os.Create(dst)
if err != nil {
return err
}
defer out.Close()
_, err = io.Copy(out, in)
if err != nil {
return err
}
return out.Close()
}
func ZipFiles(filename string, files []string) error {
newZipFile, err := os.Create(filename)
if err != nil {
return err
}
defer newZipFile.Close()
zipWriter := zip.NewWriter(newZipFile)
defer zipWriter.Close()
// Add files to zip
for _, file := range files {
zipfile, err := os.Open(file)
if err != nil {
return err
}
defer zipfile.Close()
// Get the file information
info, err := zipfile.Stat()
if err != nil {
return err
}
header, err := zip.FileInfoHeader(info)
if err != nil {
return err
}
// Using FileInfoHeader() above only uses the basename of the file. If we want
// to preserve the folder structure we can overwrite this with the full path.
filesplit := strings.Split(file, "/")
if len(filesplit) > 1 {
header.Name = filesplit[len(filesplit)-1]
} else {
header.Name = file
}
// Change to deflate to gain better compression
// see http://golang.org/pkg/archive/zip/#pkg-constants
header.Method = zip.Deflate
writer, err := zipWriter.CreateHeader(header)
if err != nil {
return err
}
if _, err = io.Copy(writer, zipfile); err != nil {
return err
}
}
return nil
}
func getAppbase(filepath string) []string {
appBase, err := ioutil.ReadFile(filepath)
if err != nil {
log.Printf("Readerror: %s", err)
os.Exit(1)
}
record := false
validLines := []string{}
for _, line := range strings.Split(string(appBase), "\n") {
if strings.Contains(line, "#STOPCOPY") {
log.Println("Stopping copy")
break
}
if record {
validLines = append(validLines, line)
}
if strings.Contains(line, "#STARTCOPY") {
log.Println("Starting copy")
record = true
}
}
return validLines
}
// Puts together ./static_baseline.py, onprem/app_sdk_app_base.py and the
// appcode in a generated_app folder based on appname+version
func stitcher(appname string, appversion string) string {
baselinefile := "static_baseline.py"
appfolder := "apps"
appbasefile := "onprem/app_sdk/app_base.py"
baseline, err := ioutil.ReadFile(baselinefile)
if err != nil {
log.Printf("Readerror: %s", err)
os.Exit(1)
}
sourceappfile := fmt.Sprintf("%s/%s/%s/src/app.py", appfolder, appname, appversion)
appfile, err := ioutil.ReadFile(sourceappfile)
if err != nil {
log.Printf("App readerror: %s", err)
os.Exit(1)
}
classname, appfile := formatAppfile(appfile)
if len(classname) == 0 {
log.Println("Failed finding classname in file.")
os.Exit(3)
}
runner := getRunner(classname)
appBase := getAppbase(appbasefile)
foldername := fmt.Sprintf("generated_apps/%s_%s", appname, appversion)
err = os.Mkdir(foldername, os.ModePerm)
if err != nil {
log.Println("Failed making temporary app folder. Probably already exists. Remaking")
os.RemoveAll(foldername)
os.MkdirAll(foldername, os.ModePerm)
}
stitched := []byte(string(baseline) + strings.Join(appBase, "\n") + string(appfile) + string(runner))
err = ioutil.WriteFile(fmt.Sprintf("%s/main.py", foldername), stitched, os.ModePerm)
if err != nil {
log.Println("Failed writing to stitched: %s", err)
os.Exit(3)
}
err = Copy(fmt.Sprintf("%s/%s/%s/requirements.txt", appfolder, appname, appversion), fmt.Sprintf("%s/requirements.txt", foldername))
if err != nil {
log.Println("Failed writing to requirement: %s", err)
os.Exit(3)
}
log.Printf("Successfully stitched files in %s/main.py", foldername)
// Zip the folder
files := []string{
fmt.Sprintf("%s/main.py", foldername),
fmt.Sprintf("%s/requirements.txt", foldername),
}
outputfile := fmt.Sprintf("%s.zip", foldername)
err = ZipFiles(outputfile, files)
if err != nil {
log.Fatal(err)
}
ctx := context.Background()
// Creates a client.
client, err := storage.NewClient(ctx)
if err != nil {
log.Printf("Failed to create client: %v", err)
os.Exit(3)
}
// Create bucket handle
bucket := client.Bucket(bucketName)
remotePath := fmt.Sprintf("apps/%s_%s.zip", appname, appversion)
err = createFileFromFile(bucket, remotePath, outputfile)
if err != nil {
log.Printf("Failed to upload to bucket: %v", err)
os.Exit(3)
}
os.Remove(outputfile)
return fmt.Sprintf("gs://%s/apps/%s_%s.zip", bucketName, appname, appversion)
}
func createFileFromFile(bucket *storage.BucketHandle, remotePath, localPath string) error {
ctx := context.Background()
// [START upload_file]
f, err := os.Open(localPath)
if err != nil {
return err
}
defer f.Close()
wc := bucket.Object(remotePath).NewWriter(ctx)
if _, err = io.Copy(wc, f); err != nil {
return err
}
if err := wc.Close(); err != nil {
return err
}
// [END upload_file]
return nil
}
// Deploy to google cloud function :)
func deployFunction(appname, localization, applocation string, environmentVariables map[string]string) error {
ctx := context.Background()
service, err := cloudfunctions.NewService(ctx)
if err != nil {
return err
}
// ProjectsLocationsListCall
projectsLocationsFunctionsService := cloudfunctions.NewProjectsLocationsFunctionsService(service)
location := fmt.Sprintf("projects/%s/locations/%s", gceProject, localization)
functionName := fmt.Sprintf("%s/functions/%s", location, appname)
cloudFunction := &cloudfunctions.CloudFunction{
AvailableMemoryMb: 128,
EntryPoint: "authorization",
EnvironmentVariables: environmentVariables,
HttpsTrigger: &cloudfunctions.HttpsTrigger{},
MaxInstances: 0,
Name: functionName,
Runtime: "python37",
SourceArchiveUrl: applocation,
}
//getCall := projectsLocationsFunctionsService.Get(fmt.Sprintf("%s/functions/function-5", location))
//resp, err := getCall.Do()
createCall := projectsLocationsFunctionsService.Create(location, cloudFunction)
_, err = createCall.Do()
if err != nil {
log.Println("Failed creating new function. Attempting patch, as it might exist already")
createCall := projectsLocationsFunctionsService.Patch(fmt.Sprintf("%s/functions/%s", location, appname), cloudFunction)
_, err = createCall.Do()
if err != nil {
log.Println("Failed patching function")
return err
}
log.Printf("Successfully patched %s to %s", appname, localization)
} else {
log.Printf("Successfully deployed %s to %s", appname, localization)
}
// FIXME - use response to define the HTTPS entrypoint. It's default to an easy one tho
return nil
}
func deployAppCloudFunc(appname string, appversion string) {
_ = os.Mkdir("generated_apps", os.ModePerm)
apikey := "eyJhbGciOiJSUzI1NiIsImtpZCI6IjYwZjQwNjBlNThkNzVmZDNmNzBiZWZmODhjNzk0YTc3NTMyN2FhMzEiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJodHRwczovL3NodWZmbGVyLmlvL2FwaS92MS93b3JrZmxvd3MvMWQ5ZDhjZTItNTY2ZS00YzNmLThhMzctNWQ2YzdkMjAwMGI1L2V4ZWN1dGUiLCJhenAiOiIxMDMwNzY3ODIwNjE0MjQ2MTg0MjIiLCJlbWFpbCI6InNjaGVkdWxlckBzaHVmZmxlLTI0MTUxNy5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJleHAiOjE1NjU1Mjc1NTEsImlhdCI6MTU2NTUyMzk1MSwiaXNzIjoiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tIiwic3ViIjoiMTAzMDc2NzgyMDYxNDI0NjE4NDIyIn0.r0EDq9fjhf_5CPTiltyfk_L3uYJp577Uy0yYPcCAl2nv50_z_oUtbWGBpQLL8gcj-NGd3g4E52Qur8k6hCMIQweLS6WAb1279vGffEoCNDfkWb3Oy-yJGP1kzwLvqFJqnHLkSWYXNWvSyWnEimW8Rryx_m1BXS5wcA8l4NIr83kS7fPZrTwjnwFSeGSThwk91DVARzapQb8r0GEgOUyHZ1aBXnV98mikzSUt-5xFKe9eMdD22YJAj0Ru-DxAxs5nOqghX4PMRysWjshjOMrlR1piPWxqAmewp8YKZDCQ5gXskpeAFBDoULT971Wsx_NCohnJsFqx1JfPS9ZYMTW2oQ"
fullAppname := fmt.Sprintf("%s-%s", strings.Replace(appname, "_", "-", -1), strings.Replace(appversion, ".", "-", -1))
locations := []string{"europe-west2"}
// Deploys the app to all locations
bucketname := stitcher(appname, appversion)
environmentVariables := map[string]string{
"FUNCTION_APIKEY": apikey,
}
for _, location := range locations {
err := deployFunction(fullAppname, location, bucketname, environmentVariables)
if err != nil {
log.Printf("Failed to deploy: %s", err)
os.Exit(3)
}
}
}
func loadYaml(fileLocation string) (WorkflowApp, error) {
action := WorkflowApp{}
yamlFile, err := ioutil.ReadFile(fileLocation)
if err != nil {
log.Printf("yamlFile.Get err: %s", err)
return WorkflowApp{}, err
}
//log.Printf(string(yamlFile))
err = yaml.Unmarshal([]byte(yamlFile), &action)
if err != nil {
return WorkflowApp{}, err
}
return action, nil
}
// FIXME - deploy to backend (YAML config)
func deployConfigToBackend(appname string, appversion string) error {
// FIXME - no static path pls
action, err := loadYaml(fmt.Sprintf("apps/%s/%s/api.yaml", appname, appversion))
if err != nil {
log.Println(err)
return err
}
action.Sharing = true
data, err := json.Marshal(action)
if err != nil {
return err
}
url := "http://localhost:5001/api/v1/workflows/apps"
client := &http.Client{}
req, err := http.NewRequest(http.MethodPut, url, bytes.NewReader(data))
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IjYwZjQwNjBlNThkNzVmZDNmNzBiZWZmODhjNzk0YTc3NTMyN2FhMzEiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJodHRwczovL3NodWZmbGVyLmlvL2FwaS92MS93b3JrZmxvd3MvMWQ5ZDhjZTItNTY2ZS00YzNmLThhMzctNWQ2YzdkMjAwMGI1L2V4ZWN1dGUiLCJhenAiOiIxMDMwNzY3ODIwNjE0MjQ2MTg0MjIiLCJlbWFpbCI6InNjaGVkdWxlckBzaHVmZmxlLTI0MTUxNy5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJleHAiOjE1NjU1Mjc1NTEsImlhdCI6MTU2NTUyMzk1MSwiaXNzIjoiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tIiwic3ViIjoiMTAzMDc2NzgyMDYxNDI0NjE4NDIyIn0.r0EDq9fjhf_5CPTiltyfk_L3uYJp577Uy0yYPcCAl2nv50_z_oUtbWGBpQLL8gcj-NGd3g4E52Qur8k6hCMIQweLS6WAb1279vGffEoCNDfkWb3Oy-yJGP1kzwLvqFJqnHLkSWYXNWvSyWnEimW8Rryx_m1BXS5wcA8l4NIr83kS7fPZrTwjnwFSeGSThwk91DVARzapQb8r0GEgOUyHZ1aBXnV98mikzSUt-5xFKe9eMdD22YJAj0Ru-DxAxs5nOqghX4PMRysWjshjOMrlR1piPWxqAmewp8YKZDCQ5gXskpeAFBDoULT971Wsx_NCohnJsFqx1JfPS9ZYMTW2oQ")
ret, err := client.Do(req)
if err != nil {
return err
}
log.Printf("Status: %s", ret.Status)
body, err := ioutil.ReadAll(ret.Body)
if err != nil {
return err
}
if ret.StatusCode != 200 {
return errors.New(fmt.Sprintf("Status %s. App probably already exists. Raw:\n%s", ret.Status, string(body)))
}
log.Println(string(body))
return nil
}
func tarDirectory(filecontext string) (io.Reader, error) {
// Create a filereader
//dockerFileReader, err := os.Open(dockerfile)
//if err != nil {
// return err
//}
//// Read the actual Dockerfile
//readDockerFile, err := ioutil.ReadAll(dockerFileReader)
//if err != nil {
// return err
//}
// Make a TAR header for the file
tarHeader := &tar.Header{
Name: filecontext,
Typeflag: tar.TypeDir,
}
// Writes the header described for the TAR file
buf := new(bytes.Buffer)
tw := tar.NewWriter(buf)
defer tw.Close()
err := tw.WriteHeader(tarHeader)
if err != nil {
return nil, err
}
dockerFileTarReader := bytes.NewReader(buf.Bytes())
return dockerFileTarReader, nil
}
func tarDir(source string, target string) (*bytes.Reader, error) {
filename := filepath.Base(source)
target = filepath.Join(target, fmt.Sprintf("%s.tar", filename))
tarfile, err := os.Create(target)
if err != nil {
return nil, err
}
defer tarfile.Close()
buf := new(bytes.Buffer)
_ = buf
tarball := tar.NewWriter(tarfile)
defer tarball.Close()
info, err := os.Stat(source)
if err != nil {
return nil, err
}
var baseDir string
if info.IsDir() {
baseDir = filepath.Base(source)
}
_ = filepath.Walk(source,
func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
header, err := tar.FileInfoHeader(info, info.Name())
if err != nil {
return err
}
if baseDir != "" {
header.Name = filepath.Join(baseDir, strings.TrimPrefix(path, source))
}
if err := tarball.WriteHeader(header); err != nil {
return err
}
if info.IsDir() {
return nil
}
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
_, err = io.Copy(tarball, file)
return nil
})
dockerFileTarReader := bytes.NewReader(buf.Bytes())
return dockerFileTarReader, nil
}
func buildImage(client *client.Client, tags []string, dockerBuildCtxDir string) error {
dockerBuildContext, err := tarDir(dockerBuildCtxDir, ".")
if err != nil {
log.Printf("Error in taring the docker root folder - %s", err.Error())
return err
}
imageBuildResponse, err := client.ImageBuild(
context.Background(),
dockerBuildContext,
types.ImageBuildOptions{
Dockerfile: "Dockerfile",
PullParent: true,
Remove: true,
Tags: tags,
NetworkMode: "host",
},
)
if err != nil {
return err
}
// Read the STDOUT from the build process
defer imageBuildResponse.Body.Close()
_, err = io.Copy(os.Stdout, imageBuildResponse.Body)
if err != nil {
return err
}
return nil
}
// FIXME - deploy to dockerhub
func deployWorker(appname, appversion string) error {
// Get dockerfile from ./apps/appname/appversion/Dockerfile
client, err := client.NewEnvClient()
if err != nil {
return err
}
tags := []string{fmt.Sprintf("%s-%s", appname, appversion)}
err = buildImage(client, tags, fmt.Sprintf("./apps/%s/%s", appname, appversion))
if err != nil {
log.Printf("Build error: %s", err)
return err
}
return nil
}
// Deploys all cloud functions. Onprem thooo :(
func deployAll() {
allapps := []string{
"hoxhunt",
"secureworks",
"servicenow",
"lastline",
"netcraft",
"misp",
"email",
"testing",
"http",
"recordedfuture",
"passivetotal",
"carbon_black",
"thehive",
"cortex",
"splunk",
}
for _, appname := range allapps {
appversion := "1.0.0"
err := deployConfigToBackend(appname, appversion)
if err != nil {
log.Printf("Failed uploading config: %s", err)
continue
}
deployAppCloudFunc(appname, appversion)
}
}
func main() {
deployAll()
return
appname := "testing"
appversion := "1.0.0"
err := deployConfigToBackend(appname, appversion)
if err != nil {
log.Printf("Failed uploading config: %s", err)
os.Exit(1)
}
deployAppCloudFunc(appname, appversion)
// FIXME - build and deploy to dockerhub as well :)
// Not able to work in remote directory propely... Even tried making an actual tar and checking it rofl
//err := deployWorker(appname, appversion)
//if err != nil {
// log.Printf("Failed to deploy docker worker: %s", err)
//}
}