Trigger GitHub BB with large input sizes
in progress
J
Johannes Rudolph
Problem / Use Case
When running meshStack Building Block Definitions (BBDs) with an implementation type for GitHub Action Pipelines, meshStack passes the entire
buildingBlockRun
JSON as an input into the workflow.As user-provided input and metadata grow over time, this JSON can exceed GitHub’s documented limits for
workflow_dispatch
inputs (65,535 characters total and 10 top-level input properties).Once these limits are hit, workflows start failing in non-obvious ways, even though the underlying BBD logic itself would still work.
Feature Request
To avoid hitting GitHub’s input size limits, the pipeline should receive a
buildingBlockRunUrl
that points to the meshStack API resource for the building block run instead of passing the full JSON object. This way the pipeline can retrieve the full data as needed without being constrained by GitHub’s input size limits.Value / Impact
- Avoids hitting GitHub’s hard input-size limits while keeping existing BBD workflows functional.
- Makes BBDs more robust for tenants with large or evolving user input payloads (e.g. many parameters, user permission inputs).
- Reduces intermittent or opaque GitHub errors related to input size, especially on GitHub Enterprise installations which have even lower input size limits.
J
Johannes Rudolph
Update: Handling large inputs for GitHub Actions triggers
We have reviewed the engineering constraints and GitHub Actions capabilities regarding this request. To support large input sizes without hitting GitHub's payload limits, we will proceed with the following changes:
What's coming
*
Opt-in payload reduction:
We will offer a configuration option on the Building Block Definition level to omit passing the full Building Block Run object to the GitHub Actions pipeline.*
Fetch-on-demand:
Instead of receiving all data in the trigger payload, your pipeline can fetch the full Building Block Run details using the meshStack API.*
API Support:
We will officially document the endpoint to retrieve a Building Block Run and ensure platform teams using the platform builder can add that permission to their API keys.Important Constraint
*
Sensitive Inputs:
Decryption of sensitive inputs will not be supported via this fetch-on-demand method.*
Recommendation:
We strongly recommend using GitHub Action's native secrets for managing sensitive credentials instead of passing them through meshStack inputs. This ensures better security and avoids masking issues in build logs, as workflow_dispatch
inputs cannot be natively masked as secrets anyway.Next steps
* Please let us know if this "fetch-on-demand" approach fits your use case and implementation constraints
* We hope to make a preview of this feature available within the next two meshStack releases
J
Johannes Rudolph
Hey all, I want to provide a quick update on this feature. We have a development preview available internally but unfortunately discovered a few last minute issues in our testing, which is why we have decided not to enable this feature in this week's release yet. I'll keep you posted on our progress for testing this.
One thing we are currently also investigating is the "upgrade" process as unfortunately GitHub actions does not allow sending undeclared inputs to a workflow dispatch event, so there will definitely be "synchronous" changes required to your workflow files to upgrade, but this can be done by pinning git refs in your building block definition versions. We will provide more documentation for this when the feature becomes generally available.
This post was marked as
in progress
Max Wipplinger
Johannes Rudolph
This seems like a good solution for that issue. The secret constraint makes sense and should be best practice anyway.
J
Johannes Rudolph
marked this post as
planned
J
Johannes Rudolph
Update: Handling large inputs for GitHub Actions triggers
We have reviewed the engineering constraints and GitHub Actions capabilities regarding this request. To support large input sizes without hitting GitHub's payload limits, we will proceed with the following changes:
What's coming
*
Opt-in payload reduction:
We will offer a configuration option on the Building Block Definition level to omit passing the full Building Block Run object to the GitHub Actions pipeline.*
Fetch-on-demand:
Instead of receiving all data in the trigger payload, your pipeline can fetch the full Building Block Run details using the meshStack API.*
API Support:
We will officially document the endpoint to retrieve a Building Block Run and ensure platform teams using the platform builder can add that permission to their API keys.Important Constraint
*
Sensitive Inputs:
Decryption of sensitive inputs will not be supported via this fetch-on-demand method.*
Recommendation:
We strongly recommend using GitHub Action's native secrets for managing sensitive credentials instead of passing them through meshStack inputs. This ensures better security and avoids masking issues in build logs, as workflow_dispatch
inputs cannot be natively masked as secrets anyway.Next steps
* Please let us know if this "fetch-on-demand" approach fits your use case and implementation constraints
* We hope to make a preview of this feature available within the next two meshStack releases