Commit Graph

793 Commits

Author SHA1 Message Date
Frikky 438b1531f8 Removed static ip for tenzir and instead rely on dhcp in predefined network 2025-11-07 16:24:45 +01:00
Frikky 2077e8beef Added env SHUFFLE_PIPELINE_STANDALONE=true to allow for use of third-party tenzir nodes instead of local 2025-11-05 23:51:58 +01:00
yashsinghcodes 392a05100e log info over what we got from backend 2025-10-31 19:58:38 +05:30
Aditya ee300ece3c fix: no need to compress data in workers 2025-10-31 19:18:40 +05:30
Aditya 5cb0acdb71 fix: disabling compressExecution for onprem 2025-10-31 19:10:36 +05:30
Aditya a7cd4bde29 fix: no need to compress data in workers 2025-10-30 21:27:14 +05:30
Aditya 0f737540d7 fix: no need to compress data in workers 2025-10-30 21:05:54 +05:30
Aditya 0da6fa67af Merge branch 'nightly' of github.com:Shuffle/Shuffle into nightly 2025-10-29 21:06:35 +05:30
yashsinghcodes f027e6196c fix: subflow stream result returning 400 2025-10-28 15:50:49 +05:30
Frikky f638baefcf Made orborus less verbose 2025-10-24 16:54:32 +02:00
Frikky e4c290b79c Merge pull request #1841 from BogatikovM/apps_scaling
fix apps scaling for k8s shuffle
2025-10-16 20:30:39 +02:00
Frikky 696bb66c83 Merge pull request #1845 from PROCYDE/shuffle-app-protocol
feat(k8s): specify app protocol for worker and app services
2025-10-16 20:29:56 +02:00
Pascal Sthamer 45ed10405c go fmt
Signed-off-by: Pascal Sthamer <10992664+P4sca1@users.noreply.github.com>
2025-10-15 11:34:53 +02:00
Pascal Sthamer 303e83ac82 go fmt
Signed-off-by: Pascal Sthamer <10992664+P4sca1@users.noreply.github.com>
2025-10-15 11:34:09 +02:00
Pascal Sthamer 68efb7111a feat(k8s): specify app protocol for worker and app services
Signed-off-by: Pascal Sthamer <10992664+P4sca1@users.noreply.github.com>
2025-10-15 11:32:53 +02:00
Pascal Sthamer 172d0d8d80 feat(k8s): allow to mount tmp volume to apps
Signed-off-by: Pascal Sthamer <10992664+P4sca1@users.noreply.github.com>
2025-10-15 11:31:43 +02:00
Frikky 39c7b1e7d0 More default behavior orborus configs 2025-10-15 11:07:41 +02:00
Aditya d8b0d5f5b2 feat: adding log forwarding features 2025-10-14 17:36:10 +05:30
BogatikovM dde6099b90 fix apps scaling for k8s shuffle 2025-10-12 14:45:41 +04:00
yashsinghcodes 8822149e19 chore: bump shuffle-shared 2025-10-10 22:26:14 +05:30
Frikky 96f91d0e9e Orborus rebuild 2025-10-09 16:05:41 +02:00
Frikky 5bfb17c9b0 Added basic revisions for datastore keys 2025-10-07 17:13:08 +02:00
Frikky a6fa8257c5 Merge branch 'nightly' of https://github.com/shuffle/shuffle into nightly 2025-10-06 14:20:49 +02:00
Frikky 35a86c2e47 More verbose details 2025-10-06 14:20:45 +02:00
yashsinghcodes f850265c1e add a random sleep so no two worker check cache at same time 2025-10-06 16:39:55 +05:30
Frikky 758fa9ebe1 More network attachments for Orborus <-> tenzir connectivity 2025-10-06 12:46:19 +02:00
Frikky 8821bed47f Bumped to 0.9.27 shared for orborus autobuild 2025-10-06 00:20:54 +02:00
Frikky 186e9d14e0 Rebuilding with /tmp:/tmp mapping in mind 2025-10-05 23:57:32 +02:00
Frikky 7dfbe3803d Minor orborus fix 2025-10-05 21:27:37 +02:00
Frikky 7cbd9456a2 More Orborus tweaks for Tenzir 2025-10-03 14:15:45 +02:00
Frikky 35aa4f194c Merge branch 'nightly' of https://github.com/shuffle/shuffle into nightly 2025-10-03 13:42:16 +02:00
Frikky d56b631b07 Nightly orborus rebuild 2025-10-03 13:42:11 +02:00
Frikky 73f677b3b2 Merge pull request #1794 from Onder7994/worker_request_limit
Worker. Add option to set k8s resources.
2025-10-03 10:57:34 +02:00
Frikky 82fa9e8ccc Merge pull request #1825 from yashsinghcodes/main
Default k8s deployment using helm charts
2025-10-02 09:49:23 +02:00
yashsinghcodes dcc20e9c84 fix default k8 deployment 2025-10-02 01:58:12 +05:30
Frikky b039222117 Merge pull request #1820 from Shuffle/main
Syncing new fixes in main back to nightly
2025-09-28 18:09:05 +02:00
Aditya 7612b4b7bb docs: a few more ideas about auto scale 2025-09-17 17:56:57 +05:30
Frikky 5da98c1d0c Re-added SHUFFLE_BASE_IMAGE_NAME usage, and made REGISTRY+BASE IMAGE setting ensure that PullAlways is used to always use latest local image 2025-09-16 16:00:47 +02:00
Frikky fc9c01abbb Fixed opensearch indexing issue in bulk 2025-09-12 08:16:18 +02:00
Frikky a046c3a96b Made service cleanup REQUIRE CLEANUP=true. This is opposite, now that we have workers that can self-discover services. 2025-09-10 01:50:10 +02:00
Frikky 954a858933 Made default app replicas 3 instead of 1. 2025-09-10 01:47:43 +02:00
Frikky 26d10c788c Disabled the Node Scale mechanism, as it was just confusing. SHUFFLE_APP_REPLICAS and SHUFFLE_SCALE_REPLICAS are self-controlled. 2025-09-10 01:46:19 +02:00
Frikky 85fd0e1abc Made swarm init more robust 2025-09-10 01:41:33 +02:00
Frikky 61d30d9fb2 Minor print fixes to orborus/worker 2025-09-09 16:00:53 +02:00
Frikky 9676317758 Fixed the problem with nodes going haywire if they crash 2025-09-04 13:21:51 +02:00
Frikky 744d8d6b7f Fixed the issue with app builds onprem not working 2025-09-04 00:43:09 +02:00
Aditya cbe8f908bd fix: enabling ppof endpoint over DEBUG_MEMORY env 2025-09-03 19:42:56 +05:30
Frikky 67cd5a3e60 Synced all 2.1.0 fixes 2025-09-01 19:19:34 +02:00
Frikky 00384fa0ea Added debug to orborus 2025-08-29 14:56:41 +02:00
Frikky d721479122 Minor tweaks for cloud & onprem execs 2025-08-29 14:16:26 +02:00