diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index d22fd6f0..ba39c8c6 100755 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -19,10 +19,10 @@ Workflows are where the magic of Shuffle automation happens. Our current ones [a Documentation is essential to any product, and Shuffle is no exception. Documentation in Shuffle uses markdown and is located in the [shuffle-docs](https://github.com/frikky/shuffle-docs/tree/master/docs) repository. These are then loaded into Shuffle when someone visits [https://shuffler/docs/about](https://shuffler/docs/about), then cached for later use. If you make an edit, expect it on our website in about an hour. #### Frontend (ReactJS) -The frontend of Shuffle is what everyone sees when they log in. Our goal here is to make it easy to get started and keep going with Shuffle - removing any blockers from the point of accessibility. If you'd like to get started, find [an issue](https://github.com/frikky/Shuffle/issues) and check the [installation guide](https://github.com/frikky/Shuffle/blob/master/install-guide.md#local-development-installation) for setting it up locally without Docker. +The frontend of Shuffle is what everyone sees when they log in. Our goal here is to make it easy to get started and keep going with Shuffle - removing any blockers from the point of accessibility. If you'd like to get started, find [an issue](https://github.com/frikky/Shuffle/issues) and check the [installation guide](https://github.com/Shuffle/Shuffle/blob/main/.github/install-guide.md) for setting it up locally without Docker. #### Backend (Golang) -The backend of Shuffle is our REST API Server that runs in the background, handling all the API-calls in general, whether from users or apps. If you'd like to get started, find [an issue](https://github.com/frikky/Shuffle/issues) and check the [installation guide](https://github.com/frikky/Shuffle/blob/master/install-guide.md#local-development-installation) for setting it up locally without Docker. +The backend of Shuffle is our REST API Server that runs in the background, handling all the API-calls in general, whether from users or apps. If you'd like to get started, find [an issue](https://github.com/frikky/Shuffle/issues) and check the [installation guide](https://github.com/Shuffle/Shuffle/blob/main/.github/install-guide.md) for setting it up locally without Docker. #### Scaling (Golang & Python) Shuffle runs using Docker, and is built to scale. There are many areas that may revolve around scaling, but the main issues come down to how we use Docker in our [architecture](https://shuffler.io/docs/architecture). If you want to help by submitting Helm charts (K8s), Docker swarm configurations, blogposts, or talk about code changes that would help scaling - please reach out (or just start building!), and we can discuss the possibilities. Make sure to read about the architecture first :) diff --git a/.github/install-guide.md b/.github/install-guide.md index 88c7299a..1ca90c4f 100755 --- a/.github/install-guide.md +++ b/.github/install-guide.md @@ -1,7 +1,7 @@ # Installation guide Installation of Shuffle is currently only available in docker. Looking for how to update Shuffle? Check the [updating guide](https://shuffler.io/docs/configuration#updating_shuffle) -This document outlines a an introduction environment which is not scalable. [Read here](https://shuffler.io/docs/configuration#production_readiness) for information on production readiness. This also includes system requirements and configurations for Swarm or Kubernetes. +This document outlines an introduction environment which is not scalable. [Read here](https://shuffler.io/docs/configuration#production_readiness) for information on production readiness. This also includes system requirements and configurations for Swarm or Kubernetes. # Docker - *nix The Docker setup is done with docker-compose @@ -17,9 +17,10 @@ cd Shuffle 3. Fix prerequisites for the Opensearch database (Elasticsearch): ```bash -mkdir shuffle-database -sudo chown -R 1000:1000 shuffle-database -# IF you get an error using 'chown', add the user first with 'sudo useradd opensearch' +mkdir shuffle-database # Create a database folder +sudo chown -R 1000:1000 shuffle-database # IF you get an error using 'chown', add the user first with 'sudo useradd opensearch' + +sudo swapoff -a # Disable swap ``` 4. Run docker-compose. @@ -78,14 +79,14 @@ Local development is pretty straight forward with **ReactJS** and **Golang**. Th **PS: You have to stop the Backend Docker container to get this one working** -**PPS: Use the "Launch" branch when developing to get it set up easier** +**PPS: Use the "main" branch when developing to get it set up easier** ## Frontend - ReactJS /w cytoscape http://localhost:3000 - Requires [npm](https://nodejs.org/en/download/)/[yarn](https://yarnpkg.com/lang/en/docs/install/#debian-stable)/your preferred manager. Runs independently from backend. ```bash cd frontend -npm i -npm start +yarn install +yarn start ``` ## Backend - Golang @@ -106,13 +107,18 @@ Large portions of the backend is written in another repository - [shuffle-shared 2. Open the Shuffle backend's go.mod file (./shuffle/backend/go.mod) (**NOT** in shuffle-shared) 3. Change the following line to point to your directory AFTER the => ``` -//replace github.com/frikky/shuffle-shared => ../../../../git/shuffle-shared +//replace github.com/frikky/shuffle-shared => ../../shuffle-shared ``` 4. Make the changes you want, then restart the backend server! 5. With your changes made, make a pull request :fire: ## Database - Opensearch -Make sure this is running through the docker-compose, and that the backend points to it with SHUFFLE_OPENSEARCH_URL defined +Make sure this is running through the docker-compose, and that the backend points to it with SHUFFLE_OPENSEARCH_URL defined. + +So essentially, what that means is: +1. Make sure you have docker-compose installed +2. Make sure you have the docker-compose.yml file from this repository +3. Run `docker-compose up opensearch -d` ## Orborus Execution of Workflows: diff --git a/.github/workflows/dockerbuild.yaml b/.github/workflows/dockerbuild.yaml index df2cc7d2..4fd63f53 100644 --- a/.github/workflows/dockerbuild.yaml +++ b/.github/workflows/dockerbuild.yaml @@ -2,7 +2,13 @@ name: dockerbuild on: push: - branches: [1.3.0] + branches: + - main + paths: + - "**" + - "!.github/**" + - "!**.md" + - "!docker-compose.yml" jobs: main: runs-on: ubuntu-latest @@ -11,45 +17,45 @@ jobs: fail-fast: false matrix: include: - - app: frontend - path: frontend - version: nightly - experimental: true - - app: backend - path: backend - version: nightly - experimental: true - - app: app_sdk - path: backend/app_sdk - version: nightly - experimental: true - - app: orborus - path: functions/onprem/orborus - version: nightly - experimental: true - - app: worker - path: functions/onprem/worker - version: nightly - experimental: true + - app: frontend + path: frontend + version: 1.3.0 + experimental: true + - app: backend + path: backend + version: 1.3.0 + experimental: true + - app: app_sdk + path: backend/app_sdk + version: 1.3.0 + experimental: true + - app: orborus + path: functions/onprem/orborus + version: 1.3.0 + experimental: true + - app: worker + path: functions/onprem/worker + version: 1.3.0 + experimental: true steps: - name: Checkout uses: actions/checkout@v3 - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - + - name: Set up QEMU uses: docker/setup-qemu-action@v3 with: - platforms: 'amd64,arm64,arm' + platforms: "amd64,arm64,arm" - name: Login to DockerHub uses: docker/login-action@v3 with: username: ${{ secrets.DOCKERHUB_USERNAME }} password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Login to Ghcr + + - name: Login to Ghcr uses: docker/login-action@v3 with: registry: ghcr.io @@ -64,14 +70,16 @@ jobs: with: logout: false context: ${{ matrix.path }}/ - file: ${{ matrix.path }}/Dockerfile + file: ${{ matrix.path }}/Dockerfile platforms: linux/amd64,linux/arm64 push: true cache-from: type=local,src=/tmp/.buildx-cache cache-to: type=local,dest=/tmp/.buildx-cache tags: | ghcr.io/shuffle/shuffle-${{ matrix.app }}:${{ matrix.version }} + ghcr.io/shuffle/shuffle-${{ matrix.app }}:latest ${{ secrets.DOCKERHUB_USERNAME }}/shuffle-${{ matrix.app }}:${{ matrix.version }} + ${{ secrets.DOCKERHUB_USERNAME }}/shuffle-${{ matrix.app }}:latest - name: Image digest run: echo ${{ steps.docker_build.outputs.digest }} diff --git a/.gitignore b/.gitignore index 3ad770b6..29491a78 100755 --- a/.gitignore +++ b/.gitignore @@ -21,6 +21,7 @@ backend/onprem/app_sdk/apps *.exe *debug* +!**/RuntimeDebugger.jsx shuffle-database/batch_metrics_enabled.conf shuffle-database/logging_enabled.conf diff --git a/README.md b/README.md index b62fac68..188da59c 100755 --- a/README.md +++ b/README.md @@ -45,6 +45,10 @@ Please consider [sponsoring](https://github.com/sponsors/frikky) the project if ## Documentation [Documentation](https://shuffler.io/docs) can be found on [https://shuffler.io/docs](https://shuffler.io/docs) and is written here: [https://github.com/frikky/shuffle-docs](https://github.com/frikky/shuffle-docs). +### Setting up a local development environment + +Please follow the steps mentioned [here](https://github.com/Shuffle/Shuffle/blob/main/.github/install-guide.md#local-development-installation)! + ## Related repositories * OpenAPI apps: [https://github.com/frikky/security-openapis](https://github.com/frikky/security-openapis) * Documentation: [https://github.com/frikky/shuffle-docs](https://github.com/frikky/shuffle-docs) @@ -75,7 +79,7 @@ These are the main areas to contribute in: Contributing guidelines are outlined [here](https://github.com/frikky/Shuffle/blob/master/.github/CONTRIBUTING.md). ## Contributors -![ICPL logo](https://github.com/frikky/Shuffle/blob/launch/frontend/src/assets/img/icpl_logo.png) +![ICPL logo](https://github.com/Shuffle/Shuffle/blob/main/frontend/src/assets/img/icpl_logo.png) **Shuffle** @@ -113,4 +117,4 @@ Below is the folder structure with a short explanation └ docker-compose.yml # Used for deployments ``` -**It's in BETA** - [Get in touch](https://shuffler.io/contact), send a mail to [frikky@shuffler.io](mailto:frikky@shuffler.io) or poke me on twitter [@frikkylikeme](https://twitter.com/frikkylikeme) +[Get in touch](https://shuffler.io/contact), send a mail to [frikky@shuffler.io](mailto:frikky@shuffler.io) or poke me on twitter [@frikkylikeme](https://twitter.com/frikkylikeme) diff --git a/backend/README.md b/backend/README.md index 4a3e0599..1262e1e8 100755 --- a/backend/README.md +++ b/backend/README.md @@ -12,7 +12,7 @@ This folder has all parts necessary for the backend to run locally and in Docker Shuffle's backend is written in Go, with apps being python (for now). More about local development can be seen in the main README. Running the backend: -``` +```bash cd go-app -go run *.go +go run main.go docker.go walkoff.go ``` diff --git a/backend/app_sdk/app_base.py b/backend/app_sdk/app_base.py index 337c58d3..298be08f 100755 --- a/backend/app_sdk/app_base.py +++ b/backend/app_sdk/app_base.py @@ -3519,6 +3519,12 @@ class AppBase: if self.action["app_name"].lower() == "shuffle tools": timeout = 55 + timeout_env = os.getenv("SHUFFLE_APP_SDK_TIMEOUT", timeout) + try: + timeout = int(timeout_env) + except Exception as e: + self.logger.info(f"[WARNING] Failed parsing timeout to int: {e}") + #timeout = 30 try: diff --git a/backend/app_sdk/requirements.txt b/backend/app_sdk/requirements.txt index 292cca1e..cd36ec7c 100644 --- a/backend/app_sdk/requirements.txt +++ b/backend/app_sdk/requirements.txt @@ -1,4 +1,4 @@ -urllib3==1.26.5 +urllib3==1.26.18 requests==2.31.0 MarkupSafe==2.0.1 liquidpy==0.8.1 diff --git a/backend/go-app/go.mod b/backend/go-app/go.mod old mode 100755 new mode 100644 index d774a438..a69a127b --- a/backend/go-app/go.mod +++ b/backend/go-app/go.mod @@ -6,7 +6,6 @@ go 1.19 require ( cloud.google.com/go/datastore v1.11.0 - cloud.google.com/go/pubsub v1.31.0 cloud.google.com/go/storage v1.30.1 github.com/basgys/goxml2json v1.1.0 github.com/carlescere/scheduler v0.0.0-20170109141437-ee74d2f83d82 @@ -22,7 +21,6 @@ require ( github.com/shuffle/shuffle-shared v0.4.79 golang.org/x/crypto v0.9.0 google.golang.org/api v0.125.0 - google.golang.org/appengine v1.6.7 google.golang.org/grpc v1.55.0 gopkg.in/src-d/go-git.v4 v4.13.1 gopkg.in/yaml.v3 v3.0.1 @@ -98,15 +96,15 @@ require ( go.opencensus.io v0.24.0 // indirect go4.org v0.0.0-20201209231011-d4a079459e60 // indirect golang.org/x/mod v0.8.0 // indirect - golang.org/x/net v0.10.0 // indirect + golang.org/x/net v0.17.0 // indirect golang.org/x/oauth2 v0.8.0 // indirect - golang.org/x/sync v0.2.0 // indirect - golang.org/x/sys v0.8.0 // indirect - golang.org/x/term v0.8.0 // indirect - golang.org/x/text v0.9.0 // indirect + golang.org/x/sys v0.13.0 // indirect + golang.org/x/term v0.13.0 // indirect + golang.org/x/text v0.13.0 // indirect golang.org/x/time v0.3.0 // indirect golang.org/x/tools v0.6.0 // indirect golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect + google.golang.org/appengine v1.6.7 // indirect google.golang.org/genproto v0.0.0-20230530153820-e85fd2cbaebc // indirect google.golang.org/genproto/googleapis/api v0.0.0-20230530153820-e85fd2cbaebc // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20230530153820-e85fd2cbaebc // indirect diff --git a/backend/go-app/go.sum b/backend/go-app/go.sum index d3fd1d01..e55ea13a 100644 --- a/backend/go-app/go.sum +++ b/backend/go-app/go.sum @@ -39,8 +39,6 @@ cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2k cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU= -cloud.google.com/go/pubsub v1.31.0 h1:aXdyyJz90kA+bor9+6+xHAciMD5mj8v15WqFZ5E0sek= -cloud.google.com/go/pubsub v1.31.0/go.mod h1:dYmJ3K97NCQ/e4OwZ20rD4Ym3Bu8Gu9m/aJdWQjdcks= cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= @@ -50,6 +48,7 @@ cloud.google.com/go/storage v1.12.0/go.mod h1:fFLk2dp2oAhDz8QFKwqrjdJvxSp/W2g7ni cloud.google.com/go/storage v1.30.1 h1:uOdMxAs8HExqBlnLtnQyP0YkvbiDpdGShGKtx6U/oNM= cloud.google.com/go/storage v1.30.1/go.mod h1:NfxhC0UJE1aXSx7CIIbCf7y9HKT7BiccwkR7+P7gN8E= dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= +github.com/AdaLogics/go-fuzz-headers v0.0.0-20210715213245-6c3934b029d8 h1:V8krnnfGj4pV65YLUm3C0/8bl7V5Nry2Pwvy3ru/wLc= github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 h1:UQHMgLO+TxOElx5B5HZ4hJQsoJ/PvUvKRhJHDQXO8P8= github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24= @@ -66,6 +65,7 @@ github.com/Masterminds/semver v1.5.0/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF0 github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY= github.com/Microsoft/go-winio v0.6.0 h1:slsWYD/zyx7lCXoZVlvQrj0hPTM1HI4+v1sIda2yDvg= github.com/Microsoft/go-winio v0.6.0/go.mod h1:cTAf44im0RAYeL23bpB+fzCyDH2MJiz2BO69KH/soAE= +github.com/Microsoft/hcsshim v0.9.6 h1:VwnDOgLeoi2du6dAznfmspNqTiwczvjv4K7NxuY9jsY= github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ= github.com/ProtonMail/go-crypto v0.0.0-20230518184743-7afd39499903 h1:ZK3C5DtzV2nVAQTx5S5jQvMeDqWtD1By5mOoyY/xJek= github.com/ProtonMail/go-crypto v0.0.0-20230518184743-7afd39499903/go.mod h1:8TI4H3IbrackdNgv+92dI+rhpCaLqM0IfpgCgenFvRE= @@ -75,11 +75,14 @@ github.com/acomagu/bufpipe v1.0.4 h1:e3H4WUzM3npvo5uv95QuJM3cQspFNtFBzvJ2oNjKIDQ github.com/acomagu/bufpipe v1.0.4/go.mod h1:mxdxdup/WdsKVreO5GpW4+M/1CE2sMG4jeGJ2sYmHc4= github.com/adrg/strutil v0.2.3 h1:WZVn3ItPBovFmP4wMHHVXUr8luRaHrbyIuLlHt32GZQ= github.com/adrg/strutil v0.2.3/go.mod h1:+SNxbiH6t+O+5SZqIj5n/9i5yUjR+S3XXVrjEcN2mxg= +github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7 h1:uSoVVbwJiQipAclBbw+8quDsfcvFjOpI5iCf4p/cqCs= github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7/go.mod h1:6zEj6s6u/ghQa61ZWa/C2Aw3RkjiTBOix7dkqa1VLIs= github.com/algolia/algoliasearch-client-go/v3 v3.18.1 h1:FP2Xtqqs/sefR5Qluygp+jVV+juXzEdJaPrZTCDLhDQ= github.com/algolia/algoliasearch-client-go/v3 v3.18.1/go.mod h1:i7tLoP7TYDmHX3Q7vkIOL4syVse/k5VJ+k0i8WqFiJk= github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c= +github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8= github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio= github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY= github.com/aws/aws-sdk-go v1.42.27/go.mod h1:OGr6lGMAKGlG9CVrYnWYDKIyb829c6EVBRjxqjmPepc= @@ -98,6 +101,7 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.19.0/go.mod h1:BgQOMsg8av8jset59jely github.com/aws/smithy-go v1.13.5/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA= github.com/basgys/goxml2json v1.1.0 h1:4ln5i4rseYfXNd86lGEB+Vi652IsIXIvggKM/BhUKVw= github.com/basgys/goxml2json v1.1.0/go.mod h1:wH7a5Np/Q4QoECFIU8zTQlZwZkrilY0itPfecMw41Dw= +github.com/bitly/go-simplejson v0.5.0 h1:6IH+V8/tVMab511d5bn4M7EwGXZf9Hj6i2xSwkNEM+Y= github.com/bitly/go-simplejson v0.5.0/go.mod h1:cXHtHw4XUPsvGaxgjIAn8PhEWG9NfngEKAMDJEczWVA= github.com/bradfitz/gomemcache v0.0.0-20221031212613-62deef7fc822 h1:hjXJeBcAMS1WGENGqDpzvmgS43oECTx8UXq31UBu0Jw= github.com/bradfitz/gomemcache v0.0.0-20221031212613-62deef7fc822/go.mod h1:H0wQNHz2YrLsuXOZozoeDmnHXkNCRmMW0gwFWDfEZDA= @@ -131,6 +135,7 @@ github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSV github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/creack/pty v1.1.11 h1:07n33Z8lZxZ2qwegKbObQohDhXDQxiMMz1NOUGYlesw= github.com/creack/pty v1.1.11/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/cyphar/filepath-securejoin v0.2.3/go.mod h1:aPGpWjXOXUn2NCNjFvBE6aRxGGx79pTxQpKOJNYHHl4= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= @@ -147,6 +152,7 @@ github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4 github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc= +github.com/elazarl/goproxy v0.0.0-20221015165544-a0805db90819 h1:RIB4cRk+lBqKK3Oy0r2gRX4ui7tuhiZq2SuTtTCi0/0= github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= github.com/emirpasic/gods v1.12.0/go.mod h1:YfzfFFoVP/catgzJb4IKIqXjX78Ha8FMSDh3ymbK86o= github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc= @@ -163,7 +169,6 @@ github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= github.com/form3tech-oss/jwt-go v3.2.3+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= github.com/frankban/quicktest v1.11.3/go.mod h1:wRf/ReqHper53s+kmmSZizM8NamnL3IM0I9ntUbOk+k= -github.com/frikky/go-elasticsearch/v8 v8.13.1/go.mod h1:RPq0JXPQVVSFHTlPwj/go8BZ1hegRf+StaSpT2iGIoQ= github.com/frikky/kin-openapi v0.41.0/go.mod h1:ev9OZAw7Bv5p0w93j91++6a1ElPzGcCofst+kmrWsj4= github.com/frikky/kin-openapi v0.42.0 h1:d5Z6vnuQ6RnCCPIxZaDL+TH2ODLxT8abytOt+Zh+Kd0= github.com/frikky/kin-openapi v0.42.0/go.mod h1:ev9OZAw7Bv5p0w93j91++6a1ElPzGcCofst+kmrWsj4= @@ -174,10 +179,12 @@ github.com/fsouza/go-dockerclient v1.9.7/go.mod h1:vx9C32kE2D15yDSOMCDaAEIARZpDQ github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= github.com/gliderlabs/ssh v0.2.2/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0= +github.com/gliderlabs/ssh v0.3.5 h1:OcaySEmAQJgyYcArR+gGGTHCyE7nvhEMTlYY+Dp8CpY= github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI= github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic= github.com/go-git/go-billy/v5 v5.4.1 h1:Uwp5tDRkPr+l/TnbHOQzp+tmJfLceOlbVucgpTz8ix4= github.com/go-git/go-billy/v5 v5.4.1/go.mod h1:vjbugF6Fz7JIflbVpl1hJsGjSHNltrSw45YK/ukIvQg= +github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20230305113008-0c11038e723f h1:Pz0DHeFij3XFhoBRGUDPzSJ+w2UcK5/0JvF8DRI58r8= github.com/go-git/go-git/v5 v5.7.0 h1:t9AudWVLmqzlo+4bqdf7GY+46SUuRsx59SboFxkq2aE= github.com/go-git/go-git/v5 v5.7.0/go.mod h1:coJHKEOk5kUClpsNlXrUvPrDxY3w3gjHvhcZd8Fodw8= github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= @@ -254,9 +261,11 @@ github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/ github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0= github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/martian v2.1.0+incompatible h1:/CP5g8u/VJHijgedC/Legn3BAbAaWPgecwXBIDzw5no= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= +github.com/google/martian/v3 v3.3.2 h1:IqNFLAmvJOgVlpdEBiQbDc2EwKW77amAycfTuWKdfvw= github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= @@ -318,14 +327,17 @@ github.com/klauspost/compress v1.11.13 h1:eSvu8Tmq6j2psUJqJrLcWH6K3w5Dwc+qipbaA6 github.com/klauspost/compress v1.11.13/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI= github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/pty v1.1.8/go.mod h1:O1sed60cT9XZ5uDucP5qwvh+TE3NnUj51EiZO/lmSfw= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e h1:hB2xlXdHp/pmPZq0y3QnmWAArdw9PqbmotexnWx/FU8= github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= +github.com/matryer/is v1.2.0 h1:92UTHpy8CDwaJ08GqLDzhhuixiBUUD1p3AU6PHddz4A= github.com/matryer/is v1.2.0/go.mod h1:2fLPjFQM9rhQ15aVEtbuwhJinnOqrmgXPNdZsdwlWXA= github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= @@ -379,6 +391,7 @@ github.com/pjbgf/sha1cd v0.3.0/go.mod h1:nZ1rrWOcGJ5uZgEEVL1VUM9iRQiZvWdbZjkKyFz github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ= @@ -391,10 +404,8 @@ github.com/seccomp/libseccomp-golang v0.9.2-0.20220502022130-f33da4d89646/go.mod github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo= github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0= github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM= -github.com/shuffle/shuffle-shared v0.4.56 h1:Z29GZB/eipt2aA4i+6heO1V2mrUmHPdw4QN4X4b1HdU= -github.com/shuffle/shuffle-shared v0.4.56/go.mod h1:X613gbo0dT3fnYvXDRwjQZyLC+T49T2nSQOrCV5QMlI= -github.com/shuffle/shuffle-shared v0.4.59 h1:5Sv8aorgQJFZr3cCKltfycdXzp9v5zlF2l3GZXjrTEo= -github.com/shuffle/shuffle-shared v0.4.59/go.mod h1:X613gbo0dT3fnYvXDRwjQZyLC+T49T2nSQOrCV5QMlI= +github.com/shuffle/shuffle-shared v0.4.66 h1:Aw4qOp0VsVJrRzW1sJhEy4OY4fRGlFErUD5+93RXL6g= +github.com/shuffle/shuffle-shared v0.4.66/go.mod h1:X613gbo0dT3fnYvXDRwjQZyLC+T49T2nSQOrCV5QMlI= github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= github.com/sirupsen/logrus v1.8.1 h1:dJKuHgqk1NNQlqoA6BTlM1Wf9DOH3NBjQyu0h9+AZZE= @@ -406,6 +417,7 @@ github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e/go.mod h1:XV66xRDq github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= github.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= +github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/src-d/gcfg v1.4.0 h1:xXbNR5AlLSA315x2UO+fTSSAXCDf+Ar38/6oyGbDKQ4= github.com/src-d/gcfg v1.4.0/go.mod h1:p/UMsR43ujA89BJY9duynAwIpvqEujIH/jFlfL7jWoI= @@ -422,6 +434,7 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/ github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= +github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8= github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww= github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0= @@ -460,8 +473,8 @@ golang.org/x/crypto v0.0.0-20220314234659-1baeb1ce4c0b/go.mod h1:IxCIyHEi3zRg3s0 golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/crypto v0.1.0/go.mod h1:RecgLatLF4+eUMCP1PoPZQb+cVrJcOPbHkTkbkB9sbw= golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU= -golang.org/x/crypto v0.9.0 h1:LF6fAI+IutBocDJ2OT0Q1g8plpYljMZ4+lty+dsqw3g= -golang.org/x/crypto v0.9.0/go.mod h1:yrmDGqONDYtNj3tH8X9dzUun2m2lzPa9ngI6/RUPGR0= +golang.org/x/crypto v0.14.0 h1:wBqGXzWJW6m1XrIKlAH0Hs1JJ7+9KBwnIO8v66Q9cHc= +golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -541,8 +554,8 @@ golang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco= golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc= -golang.org/x/net v0.10.0 h1:X2//UzNDwYmtCLn7To6G58Wr6f5ahEAQgKNzv9Y951M= -golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg= +golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM= +golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -566,7 +579,6 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.2.0 h1:PUR+T4wwASmuSTYdKjYHI5TD22Wy5ogLU5qZCOLxBrI= -golang.org/x/sync v0.2.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -621,8 +633,8 @@ golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.8.0 h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU= -golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE= +golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= @@ -630,8 +642,8 @@ golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuX golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U= -golang.org/x/term v0.8.0 h1:n5xxQn2i3PC0yLAbjTpNT85q/Kgzcr2gIoX9OrJUols= -golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo= +golang.org/x/term v0.13.0 h1:bb+I9cTfFazGW51MZqBVmZy7+JEJMouUHTUSKVQLBek= +golang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= @@ -644,8 +656,8 @@ golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ= golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= -golang.org/x/text v0.9.0 h1:2sjJmO8cDvYveuX97RDLsxlyUxLl+GHoLxBiRdHllBE= -golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= +golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k= +golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= @@ -829,12 +841,15 @@ gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= +gopkg.in/src-d/go-billy.v4 v4.3.2 h1:0SQA1pRztfTFx2miS8sA97XvooFeNOmvUenF4o0EcVg= gopkg.in/src-d/go-billy.v4 v4.3.2/go.mod h1:nDjArDMp+XMs1aFAESLRjfGSgfvoYN0hDfzEk0GjC98= +gopkg.in/src-d/go-git-fixtures.v3 v3.5.0 h1:ivZFOIltbce2Mo8IjzUHAFoq/IylO9WHhNOAJK+LsJg= gopkg.in/src-d/go-git-fixtures.v3 v3.5.0/go.mod h1:dLBcvytrw/TYZsNTWCnkNF2DSIlzWYqTe3rJR56Ac7g= gopkg.in/src-d/go-git.v4 v4.13.1 h1:SRtFyV8Kxc0UP7aCHcijOMQGPxHSmMOPrzulQWolkYE= gopkg.in/src-d/go-git.v4 v4.13.1/go.mod h1:nx5NYcxdKxq5fpltdHnPa2Exj4Sx0EclMWZQbYDu2z8= @@ -855,6 +870,7 @@ gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk= +gotest.tools/v3 v3.0.3 h1:4AuOwCGf4lLR9u3YOe2awrHygurzhO/HeQ6laiA6Sx0= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= diff --git a/docker-compose.yml b/docker-compose.yml index c35a6138..8bde6ff6 100755 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,7 +1,7 @@ version: '3' services: frontend: - image: ghcr.io/shuffle/shuffle-frontend:nightly + image: ghcr.io/shuffle/shuffle-frontend:latest container_name: shuffle-frontend hostname: shuffle-frontend ports: @@ -15,7 +15,7 @@ services: depends_on: - backend backend: - image: ghcr.io/shuffle/shuffle-backend:nightly + image: ghcr.io/shuffle/shuffle-backend:latest container_name: shuffle-backend hostname: ${BACKEND_HOSTNAME} # Here for debugging: @@ -34,7 +34,7 @@ services: - SHUFFLE_FILE_LOCATION=/shuffle-files restart: unless-stopped orborus: - image: ghcr.io/shuffle/shuffle-orborus:nightly + image: ghcr.io/shuffle/shuffle-orborus:latest container_name: shuffle-orborus hostname: shuffle-orborus networks: diff --git a/frontend/confd/templates/nginx.conf b/frontend/confd/templates/nginx.conf index 22b98574..3bb02c27 100755 --- a/frontend/confd/templates/nginx.conf +++ b/frontend/confd/templates/nginx.conf @@ -70,27 +70,17 @@ http { try_files $uri /index.html; } - location /api/v1 { + location ~ /api/v(1|2) { proxy_pass http://{{ getenv "BACKEND_HOSTNAME" "shuffle-backend" }}:5001; proxy_buffering off; proxy_http_version 1.1; - + proxy_connect_timeout 900; proxy_send_timeout 900; proxy_read_timeout 900; send_timeout 900; } - location /api/v2 { - proxy_pass http://{{ getenv "BACKEND_HOSTNAME" "shuffle-backend" }}:5001; - proxy_buffering off; - proxy_http_version 1.1; - - proxy_connect_timeout 900; - proxy_send_timeout 900; - proxy_read_timeout 900; - send_timeout 900; - } } server { @@ -122,7 +112,7 @@ http { } # Get the hostname from environment here? - location /api/v1 { + location ~ /api/v(1|2) { proxy_pass http://{{ getenv "BACKEND_HOSTNAME" "shuffle-backend" }}:5001; proxy_buffering off; proxy_http_version 1.1; diff --git a/frontend/src/components/AppSelection.jsx b/frontend/src/components/AppSelection.jsx index 45e2b426..64c99e98 100644 --- a/frontend/src/components/AppSelection.jsx +++ b/frontend/src/components/AppSelection.jsx @@ -443,7 +443,7 @@ const AppSelection = props => { }} color="rgba(158, 158, 158, 1)" > - Select the apps you work with and we will connect the for you. + Select the apps you work with and we will connect them for you. {appButtons.map((appData, index) => { diff --git a/frontend/src/components/RuntimeDebugger.jsx b/frontend/src/components/RuntimeDebugger.jsx new file mode 100644 index 00000000..fa149570 --- /dev/null +++ b/frontend/src/components/RuntimeDebugger.jsx @@ -0,0 +1,491 @@ +import React, { useState, useEffect } from 'react'; + +import { + TextField, + Link, + Button, + CircularProgress, + Select, + MenuList, + MenuItem, + FormControl, + InputLabel, + Autocomplete, + Tooltip, + Typography, +} from '@mui/material'; + +import { toast } from "react-toastify" +import { makeStyles } from "@mui/styles"; +import theme from '../theme.jsx'; +import dayjs from 'dayjs'; +import { AdapterDayjs } from '@mui/x-date-pickers/AdapterDayjs' +import Pagination from '@mui/material/Pagination'; +import { + DatePicker, + DateTimePicker, + LocalizationProvider, +} from '@mui/x-date-pickers' + +import { + OpenInNew as OpenInNewIcon, +} from '@mui/icons-material'; + +import { DataGrid, GridColDef, GridValueGetterParams } from '@mui/x-data-grid' + +const useStyles = makeStyles({ + notchedOutline: { + borderColor: "#f85a3e !important", + }, +}); + +const RuntimeDebugger = (props) => { + const { userdata, globalUrl, } = props + + const classes = useStyles(); + + //const [workflowId, setWorkflowId] = useState(""); + //const [status, setStatus] = useState("FINISHED"); + //const [endTime, setEndTime] = useState(dayjs().subtract(0, 'day')) + //const [startTime, setStartTime] = useState(dayjs().subtract(30, 'day')) + + const [workflowId, setWorkflowId] = useState("") + const [status, setStatus] = useState("") + const [endTime, setEndTime] = useState("") + const [startTime, setStartTime] = useState("") + + const [workflow, setWorkflow] = useState({}) + const [searchLoading, setSearchLoading] = useState(false) + const [rowCursor, setCursor] = useState("") + const [rowsPerPage, setRowsPerPage] = useState(10) + const [resultRows, setResultRows] = useState([]) + const [workflows, setWorkflows] = useState([ + {"id": "", "name": "All Workflows",} + ]) + /* + [ + {id: "1", execution_id: "1", status: "FINISHED", startTimestamp: "2021-10-01 12:00:00", endTimestamp: "2021-10-01 12:00:00", workflow: {"id": "1234", "name": "what",}}, + {id: "2", execution_id: "2", status: "WAITING", startTimestamp: "2021-10-01 12:00:00", endTimestamp: "2021-10-01 12:00:00", workflow: {"id": "1234", "name": "what",}}, + {id: "3", execution_id: "3", status: "EXECUTING",startTimestamp: "2021-10-01 12:00:00", endTimestamp: "2021-10-01 12:00:00", workflow: {"id": "1234", "name": "what",}}, + {id: "4", execution_id: "4", status: "ABORTED", startTimestamp: "2021-10-01 12:00:00", endTimestamp: "2021-10-01 12:00:00", workflow: {"id": "1234", "name": "what",}}, + ]); + */ + + const submitSearch = (workflowId, status, startTime, endTime, cursor, limit) => { + + setResultRows([]) + setSearchLoading(true) + const fetchData = { + workflow_id: workflowId, + cursor: cursor, + limit: limit, + + status: status.toUpperCase(), + start_time: startTime, + end_time: endTime, + } + + fetch(`${globalUrl}/api/v1/workflows/search`, { + method: "POST", + headers: { + "Content-Type": "application/json", + }, + credentials: "include", + body: JSON.stringify(fetchData), + }) + .then((response) => response.json()) + .then((data) => { + setSearchLoading(false) + if (data.success) { + if (data.runs !== undefined && data.runs !== null && data.runs.length > 0) { + for (let key in data.runs) { + data.runs[key].id = data.runs[key].execution_id + + // make started_at field into ISO string + //data.runs[key].endTimestamp = data.runs[key].ended_at.toISOString().slice(0, 19).replace('T', ' ') + const startTimestamp = new Date(data.runs[key].started_at*1000) + data.runs[key].startTimestamp = startTimestamp.toISOString().slice(0, 19).replace('T', ' ') + const endTimestamp = new Date(data.runs[key].completed_at*1000) + data.runs[key].endTimestamp = endTimestamp.toISOString().slice(0, 19).replace('T', ' ') + } + + + // Add 20 empty rows to the end of the resultRows array + // This is to make sure that the scrollbar is always visible + setResultRows(data.runs) + } + } else { + console.error("Search error: ", data.reason) + } + }) + .catch((error) => { + setSearchLoading(false) + console.error("Error:", error); + }) + } + + const getAvailableWorkflows = () => { + fetch(globalUrl + "/api/v1/workflows", { + method: "GET", + headers: { + "Content-Type": "application/json", + Accept: "application/json", + }, + credentials: "include", + }) + .then((response) => { + if (response.status !== 200) { + console.log("Status not 200 for workflows :O!"); + return; + } + return response.json(); + }) + .then((responseJson) => { + if (responseJson !== undefined) { + var foundWorkflows = [{"id": "", "name": "All Workflows",}] + foundWorkflows.push(...responseJson) + setWorkflows(foundWorkflows) + } + }) + .catch((error) => { + console.log("Error getting workflows: " + error); + }) + } + + useEffect(() => { + getAvailableWorkflows() + + // Find workflow_id in url query + const urlParams = new URLSearchParams(window.location.search); + const workflowId = urlParams.get('workflow_id'); + if (workflowId !== undefined && workflowId !== null && workflowId !== "" && workflowId.length === 36) { + setWorkflowId(workflowId) + + // find the workflow in workflows and select it + for (let key in workflows) { + if (workflows[key].id === workflowId) { + setWorkflow(workflows[key]) + break + } + } + } + + const foundStatus = urlParams.get('status'); + if (foundStatus !== undefined && foundStatus !== null && foundStatus !== "") { + setStatus(foundStatus) + } + }, []) + + const columns: GridColDef[] = [ + { + field: 'status', + headerName: 'Status', + width: 150, + renderCell: (params) => ( + { + setStatus(params.row.status) + }}> + {params.row.status} + + ), + }, + { + field: 'workflow.name', + headerName: 'Workflow Name', + width: 250, + renderCell: (params) => ( + { + setWorkflowId(params.row.workflow.id) + + for (let key in workflows) { + if (workflows[key].id === params.row.workflow.id) { + setWorkflow(workflows[key]) + break + } + } + }}> + {params.row.workflow.name} + + ), + }, + { + field: 'workflow results', + headerName: 'Results', + width: 75, + renderCell: (params) => { + //console.log("PARAMS: ", params) + var extraItems = 0 + if (params.row.workflow.triggers !== null) { + for (let key in params.row.workflow.triggers) { + if (params.row.workflow.triggers[key].app_name === "User Input" || params.row.workflow.triggers[key].app_name === "Shuffle Workflow") { + extraItems += 1 + } + } + } + + const actionLength = params.row.workflow.actions === null ? 0 : params.row.workflow.actions.length+extraItems + const resultLength = params.row.results === null ? 0 : params.row.results.length + + //console.log("actionLength: ", actionLength, " resultLength: ", resultLength) + return ( + { + }}> + {resultLength} / {actionLength} + + ) + }, + }, + { field: 'startTimestamp', headerName: 'Start time', width: 160, }, + { field: 'endTimestamp', headerName: 'End time', width: 160, }, + { + field: 'id', + headerName: 'Explore', + width: 65, + renderCell: (params) => ( + + + + ), + }, + ] + + useEffect(() => { + // Check if the user is currently focusing a texxtfield or not + // If they are, don't submit the search + if (document.activeElement.tagName === "INPUT") { + console.log("User is focusing a textfield, not submitting search") + return + } + + submitSearch(workflowId, status, startTime, endTime, rowCursor, rowsPerPage) + }, [workflowId, status, startTime, endTime]) + + const textfieldStyle = { + marginTop: 7, + marginLeft: 5, + } + + const handleStartTimeChange = (date) => { + setStartTime(date) + } + + const handleEndTimeChange = (date) => { + setEndTime(date) + } + + const handleWorkflowSelectionUpdate = (e, isUserinput) => { + if (e.target.value === undefined || e.target.value === null || e.target.value.id === undefined) { + console.log("Returning as there's no id") + return null + } + + console.log("DATA: ", e.target.value) + + setWorkflow(e.target.value) + setWorkflowId(e.target.value.id) + + submitSearch(e.target.value.id, status, startTime, endTime, rowCursor, rowsPerPage) + } + + return ( +
+

Workflow Run Debugger

+
{ + submitSearch(workflowId, status, startTime, endTime, rowCursor, rowsPerPage) + }} style={{display: "flex", }}> + + Status + + + {/* + { + setWorkflowId(e.target.value) + }} + style={textfieldStyle} + /> + */} + { + if ( + option === undefined || + option === null || + option.name === undefined || + option.name === null + ) { + return "No Workflow Selected"; + } + + const newname = ( + option.name.charAt(0).toUpperCase() + option.name.substring(1) + ).replaceAll("_", " "); + return newname; + }} + options={workflows} + fullWidth + style={{ + backgroundColor: theme.palette.inputColor, + height: 50, + borderRadius: theme.palette.borderRadius, + marginTop: 5, + marginLeft: 5, + }} + onChange={(event, newValue) => { + console.log("Found value: ", newValue) + + var parsedinput = { target: { value: newValue } } + + // For variables + if (typeof newValue === 'string' && newValue.startsWith("$")) { + parsedinput = { + target: { + value: { + "name": newValue, + "id": newValue, + "actions": [], + "triggers": [], + } + } + } + } + + handleWorkflowSelectionUpdate(parsedinput) + }} + renderOption={(props, data, state) => { + if (data.id === workflow.id) { + data = workflow; + } + + return ( + + {data.image !== undefined && data.image !== null && data.image.length > 0 ? + {data.name} + : null} + + Choose {data.name} + + + } placement="bottom"> + { + var parsedinput = { target: { value: data } } + handleWorkflowSelectionUpdate(parsedinput) + }} + > + {data.name} + + + ) + }} + renderInput={(params) => { + return ( + + ); + }} + /> + + } + /> + } + /> + + + + +
+ { + setRowsPerPage(newPageSize) + submitSearch(workflowId, status, startTime, endTime, rowCursor, newPageSize) + }} + // event for when clicking next page + // Hide page changer + onPageChange={(params) => { + console.log("params: ", params) + }} + /> +
+
+ ) +} + +export default RuntimeDebugger; diff --git a/frontend/src/views/Docs.jsx b/frontend/src/views/Docs.jsx index da00e61d..ba56b835 100755 --- a/frontend/src/views/Docs.jsx +++ b/frontend/src/views/Docs.jsx @@ -6,11 +6,13 @@ import { useParams, useNavigate, Link } from "react-router-dom"; import { isMobile } from "react-device-detect"; import theme from '../theme.jsx'; import remarkGfm from 'remark-gfm' +import KeyboardArrowRightIcon from '@mui/icons-material/KeyboardArrowRight'; +import ExpandMoreIcon from '@mui/icons-material/ExpandMore'; import { - Grid, - TextField, - IconButton, + Grid, + TextField, + IconButton, Tooltip, Divider, Button, @@ -19,11 +21,14 @@ import { Typography, Paper, List, + Collapse, + ListItemButton, + ListItemText } from "@mui/material"; -import { - Link as LinkIcon, - Edit as EditIcon, +import { + Link as LinkIcon, + Edit as EditIcon, } from "@mui/icons-material"; const Body = { @@ -34,7 +39,7 @@ const Body = { display: "flex", height: "100%", color: "white", - position: "relative", + position: "relative", //textAlign: "center", }; @@ -57,27 +62,28 @@ const innerHrefStyle = { const Docs = (defaultprops) => { const { globalUrl, selectedDoc, serverside, serverMobile } = defaultprops; - let navigate = useNavigate(); + let navigate = useNavigate(); - // Quickfix for react router 5 -> 6 - const params = useParams(); - //var props = JSON.parse(JSON.stringify(defaultprops)) - var props = Object.assign({selected: false}, defaultprops); - props.match = {} - props.match.params = params + // Quickfix for react router 5 -> 6 + const params = useParams(); + //var props = JSON.parse(JSON.stringify(defaultprops)) + var props = Object.assign({ selected: false }, defaultprops); + props.match = {} + props.match.params = params - useEffect(() => { - //if (params["key"] === undefined) { - // navigate("/docs/about") - // return - //} - }, []) - //console.log("PARAMS: ", params) + useEffect(() => { + //if (params["key"] === undefined) { + // navigate("/docs/about") + // return + //} + }, []) + //console.log("PARAMS: ", params) const [mobile, setMobile] = useState(serverMobile === true || isMobile === true ? true : false); const [data, setData] = useState(""); const [firstrequest, setFirstrequest] = useState(true); const [list, setList] = useState([]); + const [isopen, setOpen] = useState(-1); const [, setListLoaded] = useState(false); const [anchorEl, setAnchorEl] = React.useState(null); const [headingSet, setHeadingSet] = React.useState(false); @@ -98,29 +104,34 @@ const Docs = (defaultprops) => { setAnchorEl(null); } + const handleCollapse = (index) => { + setOpen(isopen === index ? -1 : index) + }; + const SidebarPaperStyle = { backgroundColor: theme.palette.surfaceColor, overflowX: "hidden", position: "relative", - padding: 30, + paddingLeft: 15, + paddingRight: 15, paddingTop: 15, marginTop: 15, - minHeight: "80vh", + minHeight: "80vh", //height: "50vh", }; const SideBar = { - minWidth: 250, - maxWidth: 300, - left: 0, - position: "sticky", - top: 50, - minHeight: "90vh", - maxHeight: "90vh", - overflowX: "hidden", - overflowY: "auto", - zIndex: 1000, - //borderRight: "1px solid rgba(255,255,255,0.3)", + minWidth: 300, + width: "20%", + left: 0, + position: "sticky", + top: 50, + minHeight: "90vh", + maxHeight: "90vh", + overflowX: "hidden", + overflowY: "auto", + zIndex: 1000, + //borderRight: "1px solid rgba(255,255,255,0.3)", }; const fetchDocList = () => { @@ -142,7 +153,7 @@ const Docs = (defaultprops) => { } setListLoaded(true); }) - .catch((error) => {}); + .catch((error) => { }); }; const fetchDocs = (docId) => { @@ -157,16 +168,16 @@ const Docs = (defaultprops) => { .then((responseJson) => { if (responseJson.success) { setData(responseJson.reason); - if (docId === undefined) { - document.title = "Shuffle documentation introduction"; - } else { - document.title = "Shuffle " + docId + " documentation"; - } + if (docId === undefined) { + document.title = "Shuffle documentation introduction"; + } else { + document.title = "Shuffle " + docId + " documentation"; + } - if (responseJson.reason !== undefined && responseJson.reason !== null && responseJson.reason.includes("404: Not Found")) { - navigate("/docs") - return - } + if (responseJson.reason !== undefined && responseJson.reason !== null && responseJson.reason.includes("404: Not Found")) { + navigate("/docs") + return + } if (responseJson.meta !== undefined) { setSelectedMeta(responseJson.meta); @@ -226,7 +237,7 @@ const Docs = (defaultprops) => { setData("# Error\nThis page doesn't exist."); } }) - .catch((error) => {}); + .catch((error) => { }); }; if (firstrequest) { @@ -245,18 +256,18 @@ const Docs = (defaultprops) => { if (!serverside) { fetchDocList(); - //const propkey = props.match.params.key - //if (propkey === undefined) { - // navigate("/docs/about") - // return null - //} - // - if (props.match.params.key === undefined) { + //const propkey = props.match.params.key + //if (propkey === undefined) { + // navigate("/docs/about") + // return null + //} + // + if (props.match.params.key === undefined) { - } else { - console.log("DOCID: ", props.match.params.key) - fetchDocs(props.match.params.key) - } + } else { + console.log("DOCID: ", props.match.params.key) + fetchDocs(props.match.params.key) + } } } } @@ -283,7 +294,7 @@ const Docs = (defaultprops) => { .join(" ") .split("-") .join(" ") - .split("?")[0] + .split("?")[0] //console.log(name) var found = false; @@ -357,15 +368,15 @@ const Docs = (defaultprops) => { color: "rgba(255, 255, 255, 0.65)", overflow: "hidden", paddingBottom: 100, - margin: "auto", - maxWidth: "100%", - minWidth: "100%", - overflow: "hidden", - fontSize: isMobile ? "1.3rem" : "1.0rem", + margin: "auto", + maxWidth: "100%", + minWidth: "100%", + overflow: "hidden", + fontSize: isMobile ? "1.3rem" : "1.0rem", }; function OuterLink(props) { - console.log("Link: ", props.href) + console.log("Link: ", props.href) if (props.href.includes("http") || props.href.includes("mailto")) { return ( { } function CodeHandler(props) { - console.log("PROPS: ", props) + console.log("PROPS: ", props) - const propvalue = props.value !== undefined && props.value !== null ? props.value : props.children !== undefined && props.children !== null && props.children.length > 0 ? props.children[0] : "" + const propvalue = props.value !== undefined && props.value !== null ? props.value : props.children !== undefined && props.children !== null && props.children.length > 0 ? props.children[0] : "" return (
{ }} > {propvalue} + style={{ + // Wrap if larger than X + whiteSpace: "pre-wrap", + overflow: "auto", + }} + >{propvalue}
); } @@ -436,7 +447,7 @@ const Docs = (defaultprops) => { display: "flex", }} > -
+
{mobile ? null : ( { href={selectedMeta.link} style={{ textDecoration: "none", color: "#f85a3e" }} > - @@ -469,15 +480,15 @@ const Docs = (defaultprops) => {
{mobile || - selectedMeta.contributors === undefined || - selectedMeta.contributors === null ? ( + selectedMeta.contributors === undefined || + selectedMeta.contributors === null ? ( "" ) : (
{selectedMeta.contributors.slice(0, 7).map((data, index) => { return ( { //} - const CustomButton = (props) => { - const {title, icon, link} = props + const CustomButton = (props) => { + const { title, icon, link } = props - const [hover, setHover] = useState(false) + const [hover, setHover] = useState(false) - return ( - -
{ - if (link === "" || link === undefined) { - event.preventDefault() - console.log("IN CLICK!") - if (window.drift !== undefined) { - window.drift.api.startInteraction({ interactionId: 340043 }) - } else { - console.log("Couldn't find drift in window.drift and not .drift-open-chat with querySelector: ", window.drift) - } - } else { - console.log("Link defined: ", link) - } - }} onMouseOver={() => { - setHover(true) - }} - onMouseOut={() => { - setHover(false); - }} - > - {icon} - - {title} - -
- - ) - } + > +
{ + if (link === "" || link === undefined) { + event.preventDefault() + console.log("IN CLICK!") + if (window.drift !== undefined) { + window.drift.api.startInteraction({ interactionId: 340043 }) + } else { + console.log("Couldn't find drift in window.drift and not .drift-open-chat with querySelector: ", window.drift) + } + } else { + console.log("Link defined: ", link) + } + }} onMouseOver={() => { + setHover(true) + }} + onMouseOut={() => { + setHover(false); + }} + > + {icon} + + {title} + +
+ + ) + } - const DocumentationButton = (props) => { - const {item, link} = props + const DocumentationButton = (props) => { + const { item, link } = props - const [hover, setHover] = useState(false); + const [hover, setHover] = useState(false); - console.log("Link: ", link) - if (link === undefined || link === null) { - return null - } + console.log("Link: ", link) + if (link === undefined || link === null) { + return null + } - return ( - -
{ - setHover(true) - }} - onMouseOut={() => { - setHover(false); - }} - > - - {item} - -
- - ) - } + return ( + +
{ + setHover(true) + }} + onMouseOut={() => { + setHover(false); + }} + > + + {item} + +
+ + ) + } - const headerStyle = { - marginTop: 25, - } + const headerStyle = { + marginTop: 25, + } - const mainpageInfo = -
- - Documentation - -
- /> - link="https://discord.gg/B2CBzUm" /> -
+ const mainpageInfo = +
+ + Documentation + +
+ /> + link="https://discord.gg/B2CBzUm" /> +
-
- Tutorial - - Dive in. Hands-on is the best approach to see how Shuffle can transform your security operations. Our set of tutorials and videos teach you how to build your skills. Check out the getting started section to give it a go! - +
+ Tutorial + + Dive in. Hands-on is the best approach to see how Shuffle can transform your security operations. Our set of tutorials and videos teach you how to build your skills. Check out the getting started section to give it a go! + - Why Shuffle? - - Security first. We incentivize trying before buying, and give you the full set of tools you need to automate your operations. What's more is we also help you find usecases that fit your unique needs. Accessibility is key, and we intend to help every SOC globally use and share their usecases. - + Why Shuffle? + + Security first. We incentivize trying before buying, and give you the full set of tools you need to automate your operations. What's more is we also help you find usecases that fit your unique needs. Accessibility is key, and we intend to help every SOC globally use and share their usecases. + - Get help - - Our promise is to make it easier and easier to automate your operations. In some cases however, it may be good with a helping hand. That's where Shuffle's consultancy and support services come in handy. We help you build and automate your operational processes to a level you haven't seen before with the help of our usecases. - + Get help + + Our promise is to make it easier and easier to automate your operations. In some cases however, it may be good with a helping hand. That's where Shuffle's consultancy and support services come in handy. We help you build and automate your operational processes to a level you haven't seen before with the help of our usecases. + - APIs - - Learn. We're all about learning, and are continuously creating documentation and video tutorials to better understand how to get started. APIs are an extremely important part of how the internet works today, and our goal is helping every security professional learn about them. - + APIs + + Learn. We're all about learning, and are continuously creating documentation and video tutorials to better understand how to get started. APIs are an extremely important part of how the internet works today, and our goal is helping every security professional learn about them. + - Workflow building - - Build. Creating workflows has never been easier. Jump into things with our getting Started section and build to your hearts content. Workflows make it all come together, with an easy to use area. - + Workflow building + + Build. Creating workflows has never been easier. Jump into things with our getting Started section and build to your hearts content. Workflows make it all come together, with an easy to use area. + - Managing Shuffle - - Organize. Whether an organization of 1000 or 1, management tools are necessary. In Shuffle we offer full user management, MFA and single-signon options, multi-tenancy and a lot more - for free! - -
+ Managing Shuffle + + Organize. Whether an organization of 1000 or 1, management tools are necessary. In Shuffle we offer full user management, MFA and single-signon options, multi-tenancy and a lot more - for free! + +
- {/* + {/* {list.map((data, index) => { const item = data.name; @@ -696,7 +707,7 @@ const Docs = (defaultprops) => { */} - {/* + {/* { }} /> */} -
+
+ // PostDataBrowser Section const postDataBrowser = list === undefined || list === null ? null : (
@@ -741,52 +753,53 @@ const Docs = (defaultprops) => { item.charAt(0).toUpperCase() + item.substring(1).split("_").join(" ").split("-").join(" "); - const itemMatching = props.match.params.key === undefined ? false : + const itemMatching = props.match.params.key === undefined ? false : props.match.params.key.toLowerCase() === item.toLowerCase(); - //const [tocLines, setTocLines] = React.useState([]); return ( -
  • - + { setTocLines([]); fetchDocs(item); + handleCollapse(index); }} > - - > {newname} - - + {newname} + + {isopen === index ? : } + {itemMatching && - tocLines !== null && - tocLines !== undefined && - tocLines.length > 0 ? ( -
    + tocLines !== null && + tocLines !== undefined && + tocLines.length > 0 ? ( + {tocLines.map((data, index) => { - //console.log(data) return ( - {}} > - - - {data.text} - - + {data.text} + + ); })} -
    + ) : null}
    • ); @@ -794,37 +807,37 @@ const Docs = (defaultprops) => {
    -
    - {props.match.params.key === undefined ? - mainpageInfo - : -
    - - {data} - -
    - } -
    +
    + {props.match.params.key === undefined ? + mainpageInfo + : +
    + + {data} + +
    + } +
    ); - // remarkPlugins={[remarkGfm]} + // remarkPlugins={[remarkGfm]} const mobileStyle = { color: "white", @@ -882,32 +895,32 @@ const Docs = (defaultprops) => { })}
    - {props.match.params.key === undefined ? - mainpageInfo - : -
    - - {data} - -
    - } + {props.match.params.key === undefined ? + mainpageInfo + : +
    + + {data} + +
    + } { // // {imageModal} - // Padding and zIndex etc set because of footer in cloud. + // Padding and zIndex etc set because of footer in cloud. const loadedCheck = ( -
    +
    {postDataBrowser} {postDataMobile}
    diff --git a/functions/kubernetes/setup_registry.sh b/functions/kubernetes/setup_registry.sh index 7b24b3e1..be4c266f 100755 --- a/functions/kubernetes/setup_registry.sh +++ b/functions/kubernetes/setup_registry.sh @@ -23,16 +23,14 @@ mkdir -p certs # Generate CA key openssl req -newkey rsa:4096 -nodes -sha256 -keyout certs/reg.key -x509 -days 365 -out certs/reg.crt -subj "/CN=$node_ip" -# generate a random string -random_string=$(openssl rand -hex 3) +echo "Generated certs/reg.key and certs/reg.crt!" -echo "Starting docker registry with name shuffle-local-registry-$random_string.." - -docker run -d -p 5000:5000 --restart=always --name "shuffle-local-registry-$random_string" \ +echo -e "Please run:\n" +echo "docker run -d -p 5000:5000 --restart=always --name "shuffle-local-registry" \ -v $(pwd)/certs:/certs \ -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/reg.crt \ -e REGISTRY_HTTP_TLS_KEY=/certs/reg.key \ - registry:2 + registry:2" -echo "Done!" +echo -e "\nnow to start the reigstry :)!" diff --git a/functions/onprem/orborus/go.sum b/functions/onprem/orborus/go.sum index 02d7a765..70af6a61 100644 --- a/functions/onprem/orborus/go.sum +++ b/functions/onprem/orborus/go.sum @@ -656,7 +656,6 @@ gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/functions/onprem/orborus/orborus.go b/functions/onprem/orborus/orborus.go index 0cd4f2cb..e5b7dc8a 100755 --- a/functions/onprem/orborus/orborus.go +++ b/functions/onprem/orborus/orborus.go @@ -1,7 +1,7 @@ package main /* - Orborus exists to listen for new workflow executions whcih are deployed as workers. + Orborus exists to listen for new workflow executions which are deployed as workers. */ // Potential issues: @@ -475,8 +475,8 @@ func deployServiceWorkers(image string) { fmt.Sprintf("SHUFFLE_SWARM_NETWORK_NAME=%s", networkName), fmt.Sprintf("SHUFFLE_APP_REPLICAS=%d", appReplicaCnt), fmt.Sprintf("SHUFFLE_LOGS_DISABLED=%s", os.Getenv("SHUFFLE_LOGS_DISABLED")), - fmt.Sprintf("SHUFFLE_DEBUG_MEMORY=%s", os.Getenv("SHUFFLE_DEBUG_MEMORY")), - fmt.Sprintf("TZ=%s", timezone), + fmt.Sprintf("DEBUG_MEMORY=%s", os.Getenv("DEBUG_MEMORY")), + fmt.Sprintf("SHUFFLE_APP_SDK_TIMEOUT=%s", os.Getenv("SHUFFLE_APP_SDK_TIMEOUT")), }, //Hosts: []string{ // innerContainerName, diff --git a/functions/onprem/worker/worker.go b/functions/onprem/worker/worker.go index 427a4bee..fb913d16 100755 --- a/functions/onprem/worker/worker.go +++ b/functions/onprem/worker/worker.go @@ -230,6 +230,8 @@ func deployApp(cli *dockerclient.Client, image string, identifier string, env [] // log.Printf("image: %s", image) // log.Printf("identifier: %s", identifier) // log.Printf("execution: %+v", workflowExecution) + log.Printf("[DEBUG] Adding SHUFFLE_APP_SDK_TIMEOUT=%s", os.Getenv("SHUFFLE_APP_SDK_TIMEOUT")) + env = append(env, fmt.Sprintf("SHUFFLE_APP_SDK_TIMEOUT=%s", os.Getenv("SHUFFLE_APP_SDK_TIMEOUT"))) if os.Getenv("IS_KUBERNETES") == "true" {