Merge pull request #48 from frikky/dev

Fixed bugs related to CORS, app creation, deletion
This commit is contained in:
Frikky
2020-06-04 16:03:11 +09:00
committed by GitHub
27 changed files with 290 additions and 1026 deletions
+17 -88
View File
@@ -1,7 +1,7 @@
# Shuffle
[Shuffle](https://shuffler.io) is an automation platform for your security stack (SOAR). It uses Docker for scaling and OpenAPI for integrations. Shuffle has the possibility to run across multiple isolated environments and tracks progress for you.
[Shuffle](https://shuffler.io) is the platform for automation needs (SOAR). It has thousands of premade integrations and uses open frameworks like OpenAPI to ease migration. The workflow editor is based on a no-code thought process to empower non-developer, and the app creator makes you able to inegrate any platform in minutes.
**It's in BETA**, but if you want to implement Shuffle - [get in touch](https://shuffler.io/contact), send a mail to [frikky@shuffler.io](mailto:frikky@shuffler.io) or on twitter [@frikkylikeme](https://twitter.com/frikkylikeme)
**It's in BETA** - [Get in touch](https://shuffler.io/contact), send a mail to [frikky@shuffler.io](mailto:frikky@shuffler.io) or poke me on twitter [@frikkylikeme](https://twitter.com/frikkylikeme)
![Example Shuffle webhook integration](shuffle_webhook.png)
@@ -12,8 +12,8 @@
## Related repositories
* Apps: https://github.com/frikky/shuffle-apps
* Workflows: https://github.com/frikky/shuffle-workflows (empty)
* Security OpenAPI apps: https://github.com/frikky/OpenAPI-security-definitions
* Documentation: https://github.com/frikky/shuffle-docs
* OpenAPI apps: https://github.com/frikky/OpenAPI-security-definitions
## Documentation
Documentation can be found on https://shuffler.io/docs/about or in your own instance. Currently lacking:
@@ -21,113 +21,42 @@ Documentation can be found on https://shuffler.io/docs/about or in your own inst
* Updates after migrating from SaaS to open source
## Features
* Premade workflows for TheHive and MISP
* Simple workflow automation editor
* Premade apps for a number of security tools
* Simple workflow editor
* App creator for [OpenAPI](https://github.com/frikky/OpenAPI-security-definitions)
* Easy to learn Python library for custom apps
## In the works
* Premade workflows for security professionals
* Dashboard - Statistics are implemented
* Debug view for manual executions
* App versioning
## Support
Open an issue on Github, or [join the gitter chat](https://gitter.im/Shuffle-SOAR/community). For other / private requests: [frikky@shuffler.io](mailto:frikky@shuffler.io)
### Setup - Local development
Frontend - requires [npm](https://nodejs.org/en/download/)/[yarn](https://yarnpkg.com/lang/en/docs/install/#debian-stable)/your preferred manager. Runs independently from backend - edit frontend/src/App.yaml (line 44~) from window.location.origin to http://YOUR IP:5001
```bash
cd frontend
npm i
npm start
```
## Website
https://shuffler.io
Backend - API calls - requires [>=go1.13](https://golang.org/dl/)
```bash
export DATASTORE_EMULATOR_HOST=0.0.0.0:8000
cd backend/go-app
go build
go run *.go
```
## License
All modular information related to Shuffle will be under MIT (anyone can use it for whatever purpose), with Shuffle itself using AGPLv3.
Database - Datastore:
```
docker run -p 8000:8000 google/cloud-sdk gcloud beta emulators datastore start --project=shuffle --host-port 0.0.0.0:8000 --no-store-on-disk
```
Apps & App SDK: MIT
Shuffle backend: AGPLv3
Orborus - Execution of Workflows:
PS: This requires some specific environment variables.
```
cd functions/onprem/orborus
go run orborus.go
```
### Project overview
### Project overview
Below is the folder structure with a short explanation
```bash
├── README.md # :)
├── backend # Contains directly backend related code. Go with sh tests
├── README.md # What you're reading right now
├── backend # Contains backend related code.
│   ├── go-app # The backend golang webserver
│   ├── app_gen # Code for app generation outside the Shuffle platform
│ └── app_sdk # The SDK used for apps
├── frontend # Contains frontend code. ReactJS and cytoscape. Horrible code :)
├── app_gen # Contains code generation projects for OpenAPI or PythonLib -> Shuffler app
├── functions # Contains google cloud function code mainly.
│   ├── static_baseline.py # Static code used by stitcher.go to generate code
│   ├── stitcher.go # Attempts to stitch together an app - part of backend now
│   ├── onprem # Code for onprem solutions
│  │   ├── Orborus # Distributes execution locations
│  │   ├── Worker # Runs a workflow
│ │   └── App_SDK # Backend of individual apps
│   └── triggers # Custom triggers used in https://shuffler.io/workflows
└ docker-compose.yml # Used for deployments
```
# Technology
GCP was chosen because why not use the best thingies. "Serverless" \o/
```bash
├── languages
│   ├── Go # I like go, which is why go.
│   ├── Python3.7 # 3.7 specifically because of f-strings and 2.7 deprecation in 2020
│   ├── Javascript # Frontend stuff. Uses ReactJS + Cytoscape for visualization
│   ├── sh/Bash # Basic testing and some deployments
├── gcloud
│   ├── datastore # TODO: Move away from this
├── onprem
│   ├── Docker # Runs the same cloud functions. I didn't like the thought of proxies
```
# Migration
Shuffle was initially built for cloud and SaaS, and a lot broke when it was moved to local execution.
There will be a major overhaul to the backend specifically. I'm currently moving and updating the following:
- Create dockerfiles and a single runscript
- * App creator - (Cloud function -> Docker)
- * Workflows - Run workflows locally
- * App list - IMPORT EXISTING APPS
- * Dockerfiles - Load the ones that are in workflows with a new version
- * Docker-compose- Frontend, backend, db & orborus
- * Configuration - Write setup documentation - Did for docker
- * Remove orborus? Can deploy straight, but that would be weird - Won't do this yet
- * Full OpenAPI support with authentication schemes in App creator (not Oauth2 yet)
- * Change workflow name
- * User run statistics
- Extended result data usage, build json with answers, not just "from previous action"
- Workflows - IMPORT DEFAULT WORKFLOWS - Create some towards e.g. TheHive & MISP.
- Documentation - General documentation /docs rewrite
- API doc - 1. In Shuffle. 2. In e.g. python
- Fix scheduler
- Use variables in e.g. JSON body
- Add random secret to orborus
```
# 1. export DATASTORE_EMULATOR_HOST=0.0.0.0:8000
# 2. docker run -p 8000:8000 google/cloud-sdk gcloud beta emulators datastore start --project=shuffle --host-port 0.0.0.0:8000 --no-store-on-disk
```
* Mail: Use appengine and connect to sendmail
# Update for users:
* Build from scratch (docker-compose)
* Push stable builds to Dockerhub
### Migration issues:
* Some workflows where items have multiple parents don't work.
* Fix dummy.json (GCP config) - bypass this somehow.
+16 -5
View File
@@ -114,18 +114,29 @@ class AppBase:
# 1. Find the action
baseresult = ""
actionname_lower = actionname.lower()
try:
if actionname.lower() == "exec":
if actionname_lower == "exec":
baseresult = execution_data["execution_argument"]
else:
for result in execution_data["results"]:
resultlabel = result["action"]["label"].replace(" ", "_", -1).lower()
if resultlabel.lower() == actionname.lower():
if resultlabel.lower() == actionname_lower:
baseresult = result["result"]
break
print("BEFORE VARIABLES!")
if len(baseresult) == 0:
print("Variables: %s" % execution_data["workflow"]["workflow_variables"])
for variable in execution_data["workflow"]["workflow_variables"]:
variablename = variable["name"].replace(" ", "_", -1).lower()
if variablename.lower() == actionname_lower:
baseresult = variable["value"]
break
except KeyError as error:
print(f"Error: {error}")
print(f"KeyError in JSON: {error}")
print(f"After first trycatch")
@@ -187,7 +198,7 @@ class AppBase:
self.logger.debug(f"\n\nHandle static data with JSON: {data}\n\n")
actualitem = re.findall(match, data, re.MULTILINE)
self.logger.info("PARSED: %s" % actualitem)
self.logger.info("STATIC PARSED: %s" % actualitem)
if len(actualitem) > 0:
for replace in actualitem:
try:
@@ -238,7 +249,7 @@ class AppBase:
self.logger.info("Fullname: %s" % fullname)
actualitem = re.findall(match, fullname, re.MULTILINE)
self.logger.info("PARSED: %s" % actualitem)
self.logger.info("ACTION PARSED: %s" % actualitem)
if len(actualitem) > 0:
for replace in actualitem:
try:
-6
View File
@@ -1,6 +0,0 @@
## RUN
```bash
cd shaffuru/
dev_appserver.py --port=5001 --host=0.0.0.0 --enable_host_checking=false .
```
+1 -1
View File
@@ -909,7 +909,7 @@ func handleGet(swagger *openapi3.Swagger, api WorkflowApp, extraParameters []Wor
}
if param.Value.In == "path" {
//log.Printf("PATH!: %s", param.Value.Name)
log.Printf("PATH!: %s", param.Value.Name)
parameters = append(parameters, param.Value.Name)
//baseUrl = fmt.Sprintf("%s%s", baseUrl)
} else if param.Value.In == "query" {
+4 -8
View File
@@ -2158,22 +2158,18 @@ func setUser(ctx context.Context, data *User) error {
return nil
}
// Used for testing only. Shouldn't impact production.
func handleCors(resp http.ResponseWriter, request *http.Request) bool {
allowedOrigins := "http://localhost:3000"
// FIXME - this is to handle multiple frontends in test rofl
origin := request.Header["Origin"]
resp.Header().Set("Vary", "Origin")
if len(origin) > 0 {
resp.Header().Set("Access-Control-Allow-Origin", origin[0])
} else {
resp.Header().Set("Access-Control-Allow-Origin", "http://localhost:4201")
}
//resp.Header().Set("Access-Control-Allow-Origin", "http://localhost:8000")
resp.Header().Set("Access-Control-Allow-Headers", "Content-Type, Accept, X-Requested-With, remember-me")
resp.Header().Set("Access-Control-Allow-Methods", "POST, GET, PUT, DELETE")
resp.Header().Set("Access-Control-Allow-Credentials", "true")
resp.Header().Set("Access-Control-Allow-Origin", allowedOrigins)
if request.Method == "OPTIONS" {
resp.WriteHeader(200)
resp.Write([]byte("OK"))
return true
+71 -2
View File
@@ -1182,7 +1182,6 @@ func saveWorkflow(resp http.ResponseWriter, request *http.Request) {
//Actions []Action `json:"actions" datastore:"actions,noindex"`
log.Printf("Hello")
body, err := ioutil.ReadAll(request.Body)
if err != nil {
log.Printf("Failed hook unmarshaling: %s", err)
@@ -1191,7 +1190,6 @@ func saveWorkflow(resp http.ResponseWriter, request *http.Request) {
return
}
log.Printf("Hello2")
var workflow Workflow
err = json.Unmarshal([]byte(body), &workflow)
//log.Printf(string(body))
@@ -1216,12 +1214,23 @@ func saveWorkflow(resp http.ResponseWriter, request *http.Request) {
log.Println("Pre")
for _, action := range workflow.Actions {
allNodes = append(allNodes, action.ID)
if action.Environment == "" {
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "An environment for %s is required"}`, action.Label)))
return
action.IsValid = true
}
// FIXME: Have a good way of tracking errors. ID's or similar.
if !action.IsValid {
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Node %s is invalid and needs to be remade."}`, action.Label)))
return
action.IsValid = true
action.Errors = []string{}
}
newActions = append(newActions, action)
}
@@ -2653,6 +2662,65 @@ func deleteWorkflowApp(resp http.ResponseWriter, request *http.Request) {
private = true
}
q := datastore.NewQuery("workflow")
var workflows []Workflow
_, err = dbclient.GetAll(ctx, q, &workflows)
if err != nil {
resp.WriteHeader(401)
resp.Write([]byte(`{"success": false, "reason": "}`))
return
}
for _, workflow := range workflows {
found := false
newActions := []Action{}
for _, action := range workflow.Actions {
if action.AppName == app.Name && action.AppVersion == app.AppVersion {
found = true
action.Errors = append(action.Errors, "App has been deleted")
action.IsValid = false
}
newActions = append(newActions, action)
}
if found {
workflow.IsValid = false
workflow.Errors = append(workflow.Errors, fmt.Sprintf("App %s_%s has been deleted", app.Name, app.AppVersion))
workflow.Actions = newActions
for _, trigger := range workflow.Triggers {
log.Printf("TRIGGER: %#v", trigger)
//err = deleteSchedule(ctx, scheduleId)
//if err != nil {
// if strings.Contains(err.Error(), "Job not found") {
// resp.WriteHeader(200)
// resp.Write([]byte(fmt.Sprintf(`{"success": true}`)))
// } else {
// resp.WriteHeader(401)
// resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Failed stopping schedule"}`)))
// }
// return
//}
}
err = setWorkflow(ctx, workflow, workflow.ID)
if err != nil {
log.Printf("Failed setting workflow when deleting app: %s", err)
continue
} else {
log.Printf("Set %s (%s) to have errors", workflow.ID, workflow.Name)
}
}
}
//resp.WriteHeader(200)
//resp.Write([]byte(`{"success": true}`))
//return
// Not really deleting it, just removing from user cache
if private {
log.Printf("Deleting private app")
@@ -2675,6 +2743,7 @@ func deleteWorkflowApp(resp http.ResponseWriter, request *http.Request) {
return
}
} else {
log.Printf("Deleting public app")
err = DeleteKey(ctx, "workflowapp", fileId)
if err != nil {
Binary file not shown.
+2 -2
View File
@@ -1,7 +1,7 @@
version: '3'
services:
frontend:
#build: ./frontend
build: ./frontend
image: frikky/shuffle:frontend
container_name: shuffle-frontend
hostname: shuffle-frontend
@@ -26,7 +26,7 @@ services:
volumes:
- ${DB_LOCATION}:/etc/shuffle
backend:
#build: ./backend
build: ./backend
image: frikky/shuffle:backend
container_name: shuffle-backend
hostname: ${BACKEND_HOSTNAME}
+1 -1
View File
@@ -371,7 +371,7 @@ const Admin = (props) => {
<Tabs
value={curTab}
indicatorColor="primary"
textColor="primary"
textColor="white"
onChange={setConfig}
aria-label="disabled tabs example"
>
+22 -8
View File
@@ -172,7 +172,11 @@ const AngularWorkflow = (props) => {
const [workflowExecutions, setWorkflowExecutions] = React.useState([]);
const unloadText = 'Are you sure you want to leave?'
useBeforeunload(() => unloadText)
useBeforeunload(() => {
if (!lastSaved) {
return unloadText
}
})
const [elements, setElements] = useState([])
const { start, stop } = useInterval({
@@ -810,7 +814,7 @@ const AngularWorkflow = (props) => {
//console.log("ACTION: ", selectedAction)
//console.log("APP: ", selectedApp)
setSelectedAction({})
//setSelectedApp({})
setSelectedApp({})
//setSelectedTrigger({})
//setSelectedEdge({})
@@ -828,6 +832,7 @@ const AngularWorkflow = (props) => {
const onEdgeSelect = (event) => {
setRightSideBarOpen(true)
setLastSaved(false)
const triggercheck = workflow.triggers.find(trigger => trigger.id === event.target.data()["source"])
if (triggercheck === undefined) {
@@ -843,6 +848,8 @@ const AngularWorkflow = (props) => {
const onNodeSelect = (event) => {
const data = event.target.data()
console.log("NODE: ", data)
setLastSaved(false)
//console.log(data)
if (data.type === "ACTION") {
@@ -949,7 +956,7 @@ const AngularWorkflow = (props) => {
}
const onNodeAdded = (event) => {
//setLastSaved(false)
setLastSaved(false)
const node = event.target
if (node.isNode() && cy.nodes().size() === 1) {
@@ -960,6 +967,7 @@ const AngularWorkflow = (props) => {
}
const onEdgeRemoved = (event) => {
setLastSaved(false)
const edge = event.target
workflow.branches = workflow.branches.filter(a => a.id !== edge.data().id)
@@ -980,6 +988,7 @@ const AngularWorkflow = (props) => {
const onNodeRemoved = (event) => {
const node = event.target
const data = node.data()
setLastSaved(false)
//var currentnode = cy.getElementById(data.id)
//if (currentnode.length === 0) {
@@ -2556,10 +2565,16 @@ const AngularWorkflow = (props) => {
const appApiView = Object.getOwnPropertyNames(selectedAction).length > 0 && Object.getOwnPropertyNames(selectedApp).length > 0 ?
<div style={appApiViewStyle}>
<div style={{display: "flex", height: 40, marginBottom: 30}}>
<div style={{flex: "1"}}>
<h3 style={{marginBottom: "5px"}}>{selectedAction.app_name}</h3>
<div style={{display: "flex", minHeight: 40, marginBottom: 30}}>
<div style={{flex: 1}}>
<h3 style={{marginBottom: 5}}>{selectedAction.app_name}</h3>
<Link to="/docs/apps" style={{textDecoration: "none", color: "#f85a3e"}}>What are apps?</Link>
{selectedAction.errors !== null && selectedAction.errors.length > 0 ?
<div>
Errors: {selectedAction.errors.join("\n")}
</div>
: null
}
</div>
<div style={{flex: "1"}}>
<Button disabled={selectedAction.id === workflow.start} style={{zIndex: 5000, marginTop: "15px",}} color="primary" variant="outlined" onClick={(e) => {
@@ -4340,7 +4355,6 @@ const AngularWorkflow = (props) => {
return null
}
setLastSaved(false)
if (Object.getOwnPropertyNames(selectedAction).length > 0 && Object.getOwnPropertyNames(selectedApp).length > 0) {
//console.time('ACTIONSTART');
return(
@@ -4850,7 +4864,7 @@ const AngularWorkflow = (props) => {
return (
<div>
<Prompt
when={true}
when={!lastSaved}
message={unloadText}
/>
{loadedCheck}
-2
View File
@@ -42,8 +42,6 @@ var globalUrl = window.location.origin
if (window.location.protocol == "http:" && window.location.port === "3000") {
globalUrl = "http://192.168.3.6:5001"
}
console.log(window.location)
console.log(globalUrl)
const surfaceColor = "#27292D"
const inputColor = "#383B40"
+4 -7
View File
@@ -374,7 +374,6 @@ const AppCreator = (props) => {
// FIXME: Have multiple authentication options?
if (securitySchemes !== undefined) {
console.log("Am I in here?")
for (const [key, value] of Object.entries(securitySchemes)) {
if (value.scheme === "bearer") {
setAuthenticationOption("Bearer auth")
@@ -525,6 +524,8 @@ const AppCreator = (props) => {
"parameters": []
}
console.log("ACTION: ", item)
if (item.queries.length > 0) {
for (var querykey in item.queries) {
const queryitem = item.queries[querykey]
@@ -943,9 +944,7 @@ const AppCreator = (props) => {
const setActionField = (field, value) => {
currentAction[field] = value
setCurrentAction(currentAction)
//if (updater !== value) {
// setUpdater(value)
//}
console.log("ACTION: ", currentAction)
}
const bodyInfo = actionBodyRequest.includes(currentActionMethod) ?
@@ -1088,7 +1087,7 @@ const AppCreator = (props) => {
}
}
if (currentAction.paths !== paths) {
if (currentAction.paths !== paths && urlPath.length > 0) {
setActionField("paths", paths)
}
@@ -1159,10 +1158,8 @@ const AppCreator = (props) => {
}}
onBlur={e => {
// Fix basic issues in frontend. Python functions run a-zA-Z0-9_
console.log(e.target.value)
const regex = /[A-Za-z0-9 _]/g;
const found = e.target.value.match(regex);
console.log("FOUND: ", found)
if (found !== null) {
setActionField("name", found.join(""))
}
+34 -3
View File
@@ -56,6 +56,7 @@ const Apps = (props) => {
const [openApiData, setOpenApiData] = React.useState("")
const [appValidation, setAppValidation] = React.useState("")
const [loadAppsModalOpen, setLoadAppsModalOpen] = React.useState(false);
const [deleteModalOpen, setDeleteModalOpen] = React.useState(false);
const [openApiModal, setOpenApiModal] = React.useState(false);
const [openApiModalType, setOpenApiModalType] = React.useState("");
const [openApiError, setOpenApiError] = React.useState("")
@@ -352,7 +353,7 @@ const Apps = (props) => {
color="primary"
style={{marginLeft: 5, marginTop: 10}}
onClick={() => {
deleteApp(selectedApp.id)
setDeleteModalOpen(true)
}}
>
<DeleteIcon />
@@ -819,13 +820,42 @@ const Apps = (props) => {
window.location.href = "/apps/new?id="+appValidation
}
const handleGithubValidation = () => {
getSpecificApps(openApi)
setLoadAppsModalOpen(false)
}
const deleteModal = deleteModalOpen ?
<Dialog modal
open={deleteModalOpen}
onClose={() => {
setDeleteModalOpen(false)
}}
PaperProps={{
style: {
backgroundColor: surfaceColor,
color: "white",
minWidth: 500,
},
}}
>
<DialogTitle>
<div style={{textAlign: "center", color: "rgba(255,255,255,0.9)"}}>
Are you sure? <div/>Some workflows may stop working.
</div>
</DialogTitle>
<DialogContent style={{color: "rgba(255,255,255,0.65)", textAlign: "center"}}>
<Button style={{}} onClick={() => {deleteApp(selectedApp.id); setDeleteModalOpen(false)}} color="primary">
Yes
</Button>
<Button variant="outlined" style={{}} onClick={() => {setDeleteModalOpen(false)}} color="primary">
No
</Button>
</DialogContent>
</Dialog>
: null
const appsModalLoad = loadAppsModalOpen ?
<Dialog modal
open={loadAppsModalOpen}
@@ -1006,6 +1036,7 @@ const Apps = (props) => {
{appView}
{modalView}
{appsModalLoad}
{deleteModal}
</div>
:
<div>
+3
View File
@@ -3,6 +3,7 @@ import React, {useState, useEffect} from 'react';
import Paper from '@material-ui/core/Paper';
import Button from '@material-ui/core/Button';
import Divider from '@material-ui/core/Divider';
import {Link} from 'react-router-dom';
import TextField from '@material-ui/core/TextField';
@@ -51,6 +52,7 @@ const Settings = (props) => {
const boxStyle = {
flex: "1",
color: "white",
marginLeft: "10px",
marginRight: "10px",
paddingLeft: "30px",
@@ -181,6 +183,7 @@ const Settings = (props) => {
<div style={{display: "flex", marginTop: "80px"}}>
<Paper style={boxStyle}>
<h2>APIKEY</h2>
<Link to="/docs/api#authentication" style={{textDecoration: "none", color: "#f85a3e"}}>What is the API key used for?</Link>
<TextField
style={{backgroundColor: inputColor, flex: "1"}}
InputProps={{
+78 -18
View File
@@ -18,6 +18,8 @@ import CachedIcon from '@material-ui/icons/Cached';
import EditIcon from '@material-ui/icons/Edit';
import MoreVertIcon from '@material-ui/icons/MoreVert';
import PlayArrowIcon from '@material-ui/icons/PlayArrow';
import AddIcon from '@material-ui/icons/Add';
import PublishIcon from '@material-ui/icons/Publish';
//import JSONPretty from 'react-json-pretty';
//import JSONPrettyMon from 'react-json-pretty/dist/monikai'
import ReactJson from 'react-json-view'
@@ -37,6 +39,9 @@ const Workflows = (props) => {
const alert = useAlert()
var upload = ""
const [file, setFile] = React.useState("");
const [workflows, setWorkflows] = React.useState([]);
const [selectedWorkflow, setSelectedWorkflow] = React.useState({});
const [selectedExecution, setSelectedExecution] = React.useState({});
@@ -314,7 +319,7 @@ const Workflows = (props) => {
// dropdown with copy etc I guess
const WorkflowPaper = (props) => {
const { data } = props;
const { data } = props;
const [open, setOpen] = React.useState(false);
const [anchorEl, setAnchorEl] = React.useState(null);
@@ -695,10 +700,8 @@ const Workflows = (props) => {
)
}
const setNewWorkflow = () => {
if (newWorkflowName.length === 0) {
return
}
// Can create and set workflows
const setNewWorkflow = (name, description, editingWorkflow, redirect) => {
var method = "POST"
var extraData = ""
@@ -710,12 +713,12 @@ const Workflows = (props) => {
workflowdata = editingWorkflow
}
workflowdata["name"] = newWorkflowName
workflowdata["description"] = newWorkflowDescription
workflowdata["name"] = name
workflowdata["description"] = description
//console.log(workflowdata)
//return
fetch(globalUrl+"/api/v1/workflows"+extraData, {
return fetch(globalUrl+"/api/v1/workflows"+extraData, {
method: method,
headers: {
'Content-Type': 'application/json',
@@ -732,17 +735,71 @@ const Workflows = (props) => {
return response.json()
})
.then((responseJson) => {
if (method === "POST") {
if (method === "POST" && redirect) {
window.location.pathname = "/workflows/"+responseJson["id"]
} else if (!redirect) {
// Update :)
getAvailableWorkflows()
} else {
alert.info("Successfully changed basic info for workflow")
}
return responseJson
})
.catch(error => {
alert.error(error.toString())
});
}
const importFiles = (event) => {
const file = event.target.value
if (event.target.files.length > 0) {
for (var key in event.target.files) {
const file = event.target.files[key]
if (file.type !== "application/json") {
//alert.error("File has to contain json.")
continue
}
const reader = new FileReader()
// Waits for the read
reader.addEventListener('load', (event) => {
var data = reader.result
try {
data = JSON.parse(reader.result)
} catch (e) {
alert.error("Invalid JSON: "+e)
return
}
// Initialize the workflow itself
const ret = setNewWorkflow(data.name, data.description, {}, false)
.then((response) => {
if (response !== undefined) {
// SET THE FULL THING
data.id = response.id
// Actually create it
const ret = setNewWorkflow(data.name, data.description, data, false)
.then((response) => {
if (response !== undefined) {
alert.success("Successfully created "+data.name)
}
})
}
})
.catch(error => {
alert.error("Import error: "+error.toString())
});
})
// Actually reads
reader.readAsText(file)
}
}
}
const modalView = modalOpen ?
<Dialog modal
open={modalOpen}
@@ -791,7 +848,7 @@ const Workflows = (props) => {
Cancel
</Button>
<Button style={{}} disabled={newWorkflowName.length === 0} onClick={() => {
setNewWorkflow()
setNewWorkflow(newWorkflowName, newWorkflowDescription, {}, true)
setModalOpen(false)
}} color="primary">
Submit
@@ -825,13 +882,16 @@ const Workflows = (props) => {
<div style={{flex: "4"}}>
<h2>Workflows</h2>
</div>
<div style={{flex: "1", display: "flex", flexDirection: "row"}}>
<div>
<Button disabled={true} color="primary" style={{marginTop: "20px",}} variant="outlined" onClick={() => setModalOpen(true)}>Import</Button>
</div>
<div>
<Button color="primary" style={{marginTop: "20px",}} variant="outlined" onClick={() => setModalOpen(true)}>New</Button>
</div>
<div style={{marginTop: 20}}>
<Tooltip color="primary" title={"Create new workflow"} placement="top">
<Button color="primary" style={{}} variant="text" onClick={() => setModalOpen(true)}><AddIcon /></Button>
</Tooltip>
<Tooltip color="primary" title={"Import workflows"} placement="top">
<Button color="primary" style={{}} variant="text" onClick={() => upload.click()}>
<PublishIcon />
</Button>
</Tooltip>
<input hidden type="file" multiple="multiple" ref={(ref) => upload = ref} onChange={importFiles} />
</div>
</div>
<Divider style={{marginBottom: "10px", height: "1px", width: "100%", backgroundColor: dividerColor}}/>
@@ -850,7 +910,7 @@ const Workflows = (props) => {
<h2>Executions: {selectedWorkflow.name}</h2>
</div>
<div style={{flex: "1"}}>
<Button color="primary" style={{marginTop: "20px"}} variant="outlined" onClick={() => {
<Button color="primary" style={{marginTop: "20px"}} variant="text" onClick={() => {
alert.info("Refreshing executions");
getWorkflowExecution(selectedWorkflow.id)
}}>
+3 -3
View File
@@ -101,10 +101,10 @@ const data = [{
},
},
{
selector: 'node[?hasErrors]',
selector: "node[!is_valid]",
css: {
'color': '#991818',
'font-style': 'italic',
'border-color': 'red',
'border-width': '10px',
},
},
{
-3
View File
@@ -1,3 +0,0 @@
main.go
*.swo
*.swp
-17
View File
@@ -1,17 +0,0 @@
# Local testing
1. Change hook.go package to main
```bash
mv ../main.go .
go run main.go hook.go
```
# Deploy local
```bash
gcloud functions deploy webhook --runtime go111 --entry-point Authorization --trigger-http --project shuffle-241517 --memory=128 --set-env-vars=FUNCTION_APIKEY=asdasd,CALLBACKURL=shuffler.io,HOOKID=test123
```
# Build and deploy from gui
1. rm webhook.zip
2. zip webhook.zip hook.go
3. Upload to bucket https://console.cloud.google.com/storage/browser/shuffle-241517.appspot.com?project=shuffle-241517
4. Restart hook(s) (https://shuffler.io/webhooks)
-416
View File
@@ -1,416 +0,0 @@
package main
// APPS:
// apps.dev.microsoft.com
// REMOVE ACCESS:
// https://portal.office.com/account/#
// Developer:
// https://developer.microsoft.com/en-us/graph/docs/concepts/permissions_reference
// Bots:
// https://dev.botframework.com/bots
// Connectors
// https://outlook.office.com/connectors/home/login/#/new
// https://go.microsoft.com/fwlink/?linkid=857599
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"strings"
"time"
)
type Info struct {
Url string `json:"url" datastore:"url"`
Name string `json:"name" datastore:"name"`
Description string `json:"description" datastore:"description"`
}
// Actions to be done by webhooks etc
// Field is the actual field to use from json
type HookAction struct {
Type string `json:"type" datastore:"type"`
Name string `json:"name" datastore:"name"`
Id string `json:"id" datastore:"id"`
Field string `json:"field" datastore:"field"`
}
type Hook struct {
Id string `json:"id" datastore:"id"`
Info Info `json:"info" datastore:"info"`
Actions []HookAction `json:"actions" datastore:"actions"`
Type string `json:"type" datastore:"type"`
Status string `json:"status" datastore:"status"`
Running bool `json:"running" datastore:"running"`
}
type TeamsHook struct {
MembersAdded []struct {
ID string `json:"id"`
} `json:"membersAdded"`
Type string `json:"type"`
Timestamp time.Time `json:"timestamp"`
LocalTimestamp string `json:"localTimestamp"`
ID string `json:"id"`
ChannelID string `json:"channelId"`
ServiceURL string `json:"serviceUrl"`
From struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"from"`
Conversation struct {
IsGroup bool `json:"isGroup"`
ConversationType string `json:"conversationType"`
ID string `json:"id"`
TenantID string `json:"tenantId"`
} `json:"conversation"`
Recipient struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"recipient"`
ChannelData struct {
Team struct {
ID string `json:"id"`
} `json:"team"`
EventType string `json:"eventType"`
Tenant struct {
ID string `json:"id"`
} `json:"tenant"`
} `json:"channelData"`
}
var hook Hook
var baseUrl = "https://shuffler.io"
type OauthToken struct {
TokenType string `json:"token_type"`
ExpiresIn int `json:"expires_in"`
ExtExpiresIn int `json:"ext_expires_in"`
AccessToken string `json:"access_token"`
}
type TeamsResponse struct {
Conversation struct {
ID string `json:"id"`
} `json:"conversation"`
From struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"from"`
Recipient struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"recipient"`
ReplyToId string `json:"replyToId"`
Type string `json:"type"`
Text string `json:"text"`
}
// This should be in a token thingy, to be controlled in workflow
func sendRequest(token OauthToken, message TeamsHook) error {
//POST https://smba.trafficmanager.net/apis/v3/conversations/12345/activities
//Authorization: Bearer eyJhbGciOiJIUzI1Ni...
//
//(JSON-serialized Activity message goes here)
tmpData := TeamsResponse{}
tmpData.Conversation.ID = message.Conversation.ID
tmpData.From = message.Recipient
tmpData.Recipient = message.From
tmpData.ReplyToId = message.ID
tmpData.Type = "message"
tmpData.Text = "HELO"
data, err := json.Marshal(tmpData)
if err != nil {
return err
}
// /v3/conversations/{conversationId}/activities/{activityId}
fullurl := fmt.Sprintf("%sv3/conversations/%s/activities", message.ServiceURL, message.Conversation.ID)
log.Println(fullurl)
log.Println(string(data))
req, err := http.NewRequest(
http.MethodPost,
fullurl,
bytes.NewBuffer([]byte(data)),
)
req.Header.Add("Authorization", fmt.Sprintf("Bearer %s", token.AccessToken))
req.Header.Add("Content-Type", "application/json")
if err != nil {
return err
}
client := http.Client{}
res, err := client.Do(req)
if err != nil {
return err
}
log.Printf("Status: %d", res.StatusCode)
body, err := ioutil.ReadAll(res.Body)
if err != nil {
return err
}
log.Println(string(body))
return nil
}
// If you're finding this: its from a test project :)
func get_accesstoken() (OauthToken, error) {
client_id := "9a2a2a63-c63c-4487-baf0-4ff3f4873a7f"
client_secret := ":3]D6oFimiXbuV20xH?Dzu@LR*6IFVbq"
fullurl := fmt.Sprintf("https://login.microsoftonline.com/botframework.com/oauth2/v2.0/token")
data := fmt.Sprintf("grant_type=client_credentials&client_id=%s&client_secret=%s&scope=https://api.botframework.com/.default", client_id, client_secret)
log.Println(data)
req, err := http.NewRequest(
http.MethodPost,
fullurl,
bytes.NewBuffer([]byte(data)),
)
if err != nil {
return OauthToken{}, err
}
client := http.Client{}
res, err := client.Do(req)
if err != nil {
return OauthToken{}, err
}
log.Printf("Status: %d", res.StatusCode)
body, err := ioutil.ReadAll(res.Body)
if err != nil {
return OauthToken{}, err
}
token := OauthToken{}
err = json.Unmarshal(body, &token)
if err != nil {
return OauthToken{}, err
}
return token, nil
}
//func CheckTenantId(message TeamsHook) {
// fullurl := fmt.Sprintf("%s/api/v1/functions/tenants/%s", baseUrl, message.Conversation.TenantID)
// req, err := http.NewRequest(
// http.MethodPost,
// fullurl,
// bytes.NewBuffer([]byte(data)),
// )
//
// req.Header.Add("Authorization", fmt.Sprintf(`Bearer %s`, baseApikey))
// req.Header.Add("Content-Type", "application/json")
// if err != nil {
// return []string{}, err
// }
//
// client := http.Client{}
// res, err := client.Do(req)
// if err != nil {
// return []string{}, err
// }
//
// log.Printf("Status: %d", res.StatusCode)
// body, err := ioutil.ReadAll(res.Body)
// if err != nil {
// return []string{}, err
// }
//}
func Authorization(resp http.ResponseWriter, request *http.Request) {
// FIXME - don't have this here, but before loops etc
// How to keep it refreshed?
token, err := get_accesstoken()
if err != nil {
log.Printf("Failed: %s", err)
}
body, err := ioutil.ReadAll(request.Body)
if err != nil {
return
}
log.Println("Data")
log.Println(string(body))
hook := TeamsHook{}
err = json.Unmarshal(body, &hook)
if err != nil {
resp.WriteHeader(200)
resp.Write([]byte(`{"success": false}`))
return
}
// Only handle messages currently
if hook.Type != "message" {
resp.WriteHeader(200)
resp.Write([]byte(`{"success": false}`))
return
}
// Find the ORG based on the above info. How?
// MSTeams hook should have it attached somehow?
log.Printf(string(body))
//log.Printf(hook.ServiceURL)
//log.Printf(hook.ChannelID)
//log.Printf(hook.ID)
//log.Printf("%#v", hook.Conversation)
err = sendRequest(token, hook)
if err != nil {
log.Printf("Failed: %s", err)
}
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"success": true}`)))
}
func loadConfiguration(fullUrl string, apikey string) (Hook, error) {
client := &http.Client{}
req, err := http.NewRequest(
"GET",
fullUrl,
nil,
)
if err != nil {
log.Printf("Error making http request: %s", req)
return Hook{}, err
}
req.Header.Add("Authorization", fmt.Sprintf(`Bearer %s`, apikey))
req.Header.Add("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
log.Printf("Error in http request: %s", req)
return Hook{}, err
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Printf("Error reading response: %s", req)
return Hook{}, err
}
err = json.Unmarshal(body, &hook)
if err != nil {
log.Printf("Failed unmarshaling hook API", req)
return Hook{}, err
}
return hook, nil
}
// GetUserDetails - Get one user's details from randomuser.me API
func ForwardRequest(resp http.ResponseWriter, request *http.Request) error {
callbackUrl := os.Getenv("CALLBACKURL")
hookId := os.Getenv("HOOKID")
apikey := os.Getenv("FUNCTION_APIKEY")
hook, err := loadConfiguration(
fmt.Sprintf("%s/api/v1/hooks/%s", callbackUrl, hookId),
apikey,
)
log.Println("Done loading!")
if err != nil {
return err
}
log.Printf("%#v", hook)
// Find all things to execute
workflowUrls := []string{}
for _, item := range hook.Actions {
if item.Type == "" {
log.Printf("CONTINUE AAS EMPTY ITEM: %#v", item)
continue
}
if item.Type == "workflow" {
workflowUrls = append(workflowUrls, item.Id)
}
}
if len(workflowUrls) == 0 {
return errors.New("No actions to do yet")
}
log.Printf("Should send data to the following: %s", strings.Join(workflowUrls, ", "))
randomUserClient := http.Client{
Timeout: time.Second * 3,
}
body, err := ioutil.ReadAll(request.Body)
if err != nil {
return err
}
// Prepare data
type arg struct {
ExecutionArgument string `json:"execution_argument"`
}
data := arg{
ExecutionArgument: string(body),
}
newjson, err := json.Marshal(data)
if err != nil {
return err
}
// Loop all executions to run
for _, item := range workflowUrls {
fullUrl := fmt.Sprintf("%s/api/v1/workflows/%s/execute", callbackUrl, item)
log.Printf("Sending data to %s", fullUrl)
req, err := http.NewRequest(
http.MethodPost,
fullUrl,
bytes.NewBuffer(newjson),
)
req.Header.Add("Authorization", fmt.Sprintf(`Bearer %s`, apikey))
req.Header.Add("Content-Type", "application/json")
if err != nil {
return err
}
res, err := randomUserClient.Do(req)
if err != nil {
return err
}
log.Printf("Status: %d", res.StatusCode)
body, err := ioutil.ReadAll(res.Body)
if err != nil {
return err
}
log.Printf(string(body))
}
//log.Println(string(newbody))
return nil
}
-41
View File
@@ -1,41 +0,0 @@
package main
import (
"log"
"net/http"
"os"
"github.com/gorilla/handlers"
"github.com/gorilla/mux"
)
func webhook() {
// FIXME - remove static
port := ":8080"
baseFilePath := "/"
mux := mux.NewRouter()
mux.SkipClean(true)
// FIXME - Add path for updating the hook? Can be a specific POST requeuest from backend
mux.HandleFunc(baseFilePath, Authorization).Methods("POST")
mux.HandleFunc("/test", Authorization).Methods("POST")
handlers.LoggingHandler(os.Stdout, mux)
loggedRouter := handlers.LoggingHandler(os.Stdout, mux)
log.Printf("Starting on http://localhost%s", port)
err := http.ListenAndServe(
port,
loggedRouter,
)
if err != nil {
log.Fatal("ListenAndServer: ", err)
}
}
func main() {
webhook()
}
-50
View File
@@ -1,50 +0,0 @@
{
"$schema": "https://developer.microsoft.com/en-us/json-schemas/teams/v1.5/MicrosoftTeams.schema.json",
"manifestVersion": "1.5",
"version": "1.0.0",
"id": "9a2a2a63-c63c-4487-baf0-4ff3f4873a7f",
"packageName": "com.example.myapp",
"devicePermissions" : [],
"developer": {
"name": "@frikkylikeme",
"websiteUrl": "https://shuffler.io/",
"privacyUrl": "https://shuffler.io/privacy",
"termsOfUseUrl": "https://shuffler.io/tos"
},
"localizationInfo": {
"defaultLanguageTag": "en-us"
},
"name": {
"short": "Shuffle",
"full": "Shuffle"
},
"description": {
"short": "Shuffle is a workflow automation platform",
"full": "Shuffle is a workflow automation platform. Find more info at https://shuffler.io"
},
"icons": {
"outline": "outline.png",
"color": "color.png"
},
"accentColor": "#15202b",
"bots": [
{
"botId": "9a2a2a63-c63c-4487-baf0-4ff3f4873a7f",
"needsChannelSelector": false,
"isNotificationOnly": false,
"scopes": [ "team", "personal", "groupchat" ],
"supportsFiles": false,
"commandLists": [
{
"scopes": [ "team", "groupchat", "personal" ],
"commands": [
{
"title": "test",
"description": "THIS IS FOR TESTING"
}
]
}
]
}
]
}
-35
View File
@@ -1,35 +0,0 @@
curl -XPOST http://localhost:8080 -d '{
"membersAdded": [
{
"id": "28:f5d48856-5b42-41a0-8c3a-c5f944b679b0"
}
],
"type": "conversationUpdate",
"timestamp": "2017-02-23T19:38:35.312Z",
"localTimestamp": "2017-02-23T12:38:35.312-07:00",
"id": "f:5f85c2ad",
"channelId": "msteams",
"serviceUrl": "https://smba.trafficmanager.net/amer-client-ss.msg/",
"from": {
"id": "29:1I9Is_Sx0OIy2rQ7Xz1lcaPKlO9eqmBRTBuW6XzkFtcjqxTjPaCMij8BVMdBcL9L_RwWNJyAHFQb0TRzXgyQvA"
},
"conversation": {
"isGroup": true,
"conversationType": "channel",
"id": "19:efa9296d959346209fea44151c742e73@thread.skype"
},
"recipient": {
"id": "28:f5d48856-5b42-41a0-8c3a-c5f944b679b0",
"name": "SongsuggesterBot"
},
"channelData": {
"team": {
"id": "19:efa9296d959346209fea44151c742e73@thread.skype"
},
"eventType": "teamMemberAdded",
"tenant": {
"id": "72f988bf-86f1-41af-91ab-2d7cd011db47"
}
}
}'
#{"type":"message","id":"4oN7bHB4dit7scwHygF1pf-h|0000000","timestamp":"2019-09-06T15:21:21.9035613Z","serviceUrl":"https://webchat.botframework.com/","channelId":"webchat","from":{"id":"4ccfb6b9-5755-426e-914d-641dd74f5e0f"},"conversation":{"id":"4oN7bHB4dit7scwHygF1pf-h"},"recipient":{"id":"Shuffle@qKw6tMx9fE8","name":"Shuffler"},"textFormat":"plain","locale":"en-US","text":"hi","entities":[{"type":"ClientCapabilities","requiresBotState":true,"supportsListening":true,"supportsTts":true}],"channelData":{"clientActivityID":"15677832808420.ishosdmdfbd"}}
-3
View File
@@ -1,3 +0,0 @@
main.go
*.swo
*.swp
-17
View File
@@ -1,17 +0,0 @@
# Local testing
1. Change hook.go package to main
```bash
mv ../main.go .
go run main.go hook.go
```
# Deploy local
```bash
gcloud functions deploy webhook --runtime go111 --entry-point Authorization --trigger-http --project shuffler --memory=128 --set-env-vars=FUNCTION_APIKEY=asdasd,CALLBACKURL=shuffler.io,HOOKID=test123
```
# Build and deploy from gui
1. rm webhook.zip
2. zip webhook.zip hook.go
3. Upload to bucket https://console.cloud.google.com/storage/browser/shuffler.appspot.com?project=shuffler
4. Restart hook(s) (https://shuffler.io/webhooks)
-249
View File
@@ -1,249 +0,0 @@
package function
// BOTS
// https://dev.botframework.com/bots/channels?id=Shuffle
// APPS:
// apps.dev.microsoft.com
// REMOVE ACCESS:
// https://portal.office.com/account/#
// Developer:
// https://developer.microsoft.com/en-us/graph/docs/concepts/permissions_reference
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"strings"
"time"
)
type Info struct {
Url string `json:"url" datastore:"url"`
Name string `json:"name" datastore:"name"`
Description string `json:"description" datastore:"description"`
}
// Actions to be done by webhooks etc
// Field is the actual field to use from json
type HookAction struct {
Type string `json:"type" datastore:"type"`
Name string `json:"name" datastore:"name"`
Id string `json:"id" datastore:"id"`
Field string `json:"field" datastore:"field"`
}
type Hook struct {
Id string `json:"id" datastore:"id"`
Info Info `json:"info" datastore:"info"`
Actions []HookAction `json:"actions" datastore:"actions"`
Type string `json:"type" datastore:"type"`
Status string `json:"status" datastore:"status"`
Running bool `json:"running" datastore:"running"`
}
var hook Hook
func Authorization(resp http.ResponseWriter, request *http.Request) {
apikey := os.Getenv("FUNCTION_APIKEY")
callbackUrl := os.Getenv("CALLBACKURL")
hookId := os.Getenv("HOOKID")
if len(apikey) == 0 {
log.Println("Env FUNCTION_APIKEY not set")
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Internal error"}`)))
return
}
if len(callbackUrl) == 0 {
log.Println("Env CALLBACKURL not set")
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Internal error"}`)))
return
}
if len(hookId) == 0 {
log.Println("Env HOOKID not set")
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Internal error"}`)))
return
}
authorization := request.Header.Get("Authorization")
if len(authorization) == 0 {
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Authorization header required"}`)))
return
}
if !strings.HasPrefix(authorization, "Bearer") {
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Authorization header must start with Bearer"}`)))
return
}
apikeyCheck := strings.Split(authorization, " ")
if len(apikeyCheck) != 2 {
log.Println("Length is not 2 for apikey: %s vs %s", apikeyCheck[1], apikey)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Invalid Apikey"}`)))
return
}
if apikeyCheck[1] != apikey {
log.Printf("Apikeys are not equal. Failed authentication.")
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "Invalid Apikey"}`)))
return
}
err := ForwardRequest(resp, request)
if err != nil {
log.Printf("Error: %s", err)
resp.WriteHeader(401)
resp.Write([]byte(fmt.Sprintf(`{"success": false, "reason": "%s"}`, err)))
return
}
log.Println("Success?")
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"success": true}`)))
}
func loadConfiguration(fullUrl string, apikey string) (Hook, error) {
client := &http.Client{}
req, err := http.NewRequest(
"GET",
fullUrl,
nil,
)
if err != nil {
log.Printf("Error making http request: %s", req)
return Hook{}, err
}
req.Header.Add("Authorization", fmt.Sprintf(`Bearer %s`, apikey))
req.Header.Add("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
log.Printf("Error in http request: %s", req)
return Hook{}, err
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Printf("Error reading response: %s", req)
return Hook{}, err
}
err = json.Unmarshal(body, &hook)
if err != nil {
log.Printf("Failed unmarshaling hook API", req)
return Hook{}, err
}
return hook, nil
}
// GetUserDetails - Get one user's details from randomuser.me API
func ForwardRequest(resp http.ResponseWriter, request *http.Request) error {
callbackUrl := os.Getenv("CALLBACKURL")
hookId := os.Getenv("HOOKID")
apikey := os.Getenv("FUNCTION_APIKEY")
hook, err := loadConfiguration(
fmt.Sprintf("%s/api/v1/hooks/%s", callbackUrl, hookId),
apikey,
)
log.Println("Done loading!")
if err != nil {
return err
}
log.Printf("%#v", hook)
// Find all things to execute
workflowUrls := []string{}
for _, item := range hook.Actions {
if item.Type == "" {
log.Printf("CONTINUE AAS EMPTY ITEM: %#v", item)
continue
}
if item.Type == "workflow" {
workflowUrls = append(workflowUrls, item.Id)
}
}
if len(workflowUrls) == 0 {
return errors.New("No actions to do yet")
}
log.Printf("Should send data to the following: %s", strings.Join(workflowUrls, ", "))
randomUserClient := http.Client{
Timeout: time.Second * 3,
}
body, err := ioutil.ReadAll(request.Body)
if err != nil {
return err
}
// Prepare data
type arg struct {
ExecutionArgument string `json:"execution_argument"`
}
data := arg{
ExecutionArgument: string(body),
}
newjson, err := json.Marshal(data)
if err != nil {
return err
}
// Loop all executions to run
for _, item := range workflowUrls {
fullUrl := fmt.Sprintf("%s/api/v1/workflows/%s/execute", callbackUrl, item)
log.Printf("Sending data to %s", fullUrl)
req, err := http.NewRequest(
http.MethodPost,
fullUrl,
bytes.NewBuffer(newjson),
)
req.Header.Add("Authorization", fmt.Sprintf(`Bearer %s`, apikey))
req.Header.Add("Content-Type", "application/json")
if err != nil {
return err
}
res, err := randomUserClient.Do(req)
if err != nil {
return err
}
log.Printf("Status: %d", res.StatusCode)
body, err := ioutil.ReadAll(res.Body)
if err != nil {
return err
}
log.Printf(string(body))
}
//log.Println(string(newbody))
return nil
}
-39
View File
@@ -1,39 +0,0 @@
package main
import (
"log"
"net/http"
"os"
"github.com/gorilla/handlers"
"github.com/gorilla/mux"
)
func webhook() {
// FIXME - remove static
port := ":8080"
baseFilePath := "/"
mux := mux.NewRouter()
mux.SkipClean(true)
// FIXME - Add path for updating the hook? Can be a specific POST requeuest from backend
mux.HandleFunc(baseFilePath, Authorization).Methods("POST")
handlers.LoggingHandler(os.Stdout, mux)
loggedRouter := handlers.LoggingHandler(os.Stdout, mux)
err := http.ListenAndServe(
port,
loggedRouter,
)
if err != nil {
log.Fatal("ListenAndServer: ", err)
}
}
func main() {
webhook()
}
+34 -2
View File
@@ -21,14 +21,46 @@ cd Shuffle
docker-compose up -d
```
## After installation
### After installation
1. After installation, go to http://localhost:3001/adminsetup (or your servername)
2. Now set up your admin account (username & password). Shuffle doesn't have a default username and password.
![Admin account setup](shuffle_adminaccount.png)
## Useful info
### Useful info
* The server is available on http://localhost:3001 (or your servername)
* Further configurations can be done in docker-compose.yml and .env.
* Default database location is /etc/shuffle
## Local development installation
Frontend - requires [npm](https://nodejs.org/en/download/)/[yarn](https://yarnpkg.com/lang/en/docs/install/#debian-stable)/your preferred manager. Runs independently from backend - edit frontend/src/App.yaml (line 44~) from window.location.origin to http://YOUR IP:5001
```bash
cd frontend
npm i
npm start
```
Backend - API calls - requires [>=go1.13](https://golang.org/dl/)
```bash
export DATASTORE_EMULATOR_HOST=0.0.0.0:8000
cd backend/go-app
go build
go run *.go
```
Database - Datastore:
```
docker run -p 8000:8000 google/cloud-sdk gcloud beta emulators datastore start --project=shuffle --host-port 0.0.0.0:8000 --no-store-on-disk
```
Orborus - Execution of Workflows:
PS: This requires some specific environment variables.
```
cd functions/onprem/orborus
go run orborus.go
```