-
Notifications
You must be signed in to change notification settings - Fork 53
Optimize webhooks JSON with compact structure and string interning #232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
bfb514f to
4dacd2c
Compare
4dacd2c to
27aec5d
Compare
| run: cd languageservice && npm test -- --testPathPattern=eventPayloads | ||
| - name: Verify validation tests ran | ||
| run: | | ||
| if [ ! -f languageservice/src/context-providers/events/webhooks.full.validation-complete ]; then |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the test is normally skipped when the full webhooks file doesn't exist, I made the test write a marker file. With the marker file, we can be sure the test actually ran.
The full webhooks file (unoptimized) is written above by the script npm run update-webhooks
| exit 1 | ||
| fi | ||
| validate-webhooks: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added this job to validate the optimized webhooks JSON is equivalent to the full webhooks JSON file.
Refer changes to the file eventPayloads.test.ts in this PR
27aec5d to
8fbd093
Compare
| @@ -1,310 +0,0 @@ | |||
| import {promises as fs} from "fs"; | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I renamed this file to update-webhooks.ts but there were also significant updates anyway
| @@ -1,7 +1,8 @@ | |||
| import {data, DescriptionDictionary} from "@actions/expressions"; | |||
|
|
|||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
eventsPayloads.ts reads the compact webhook JSON files which contains the keys and descriptions for each GitHub event payload. This information is used to validate github.event.* syntax and provide descriptions on hover.
A unit test that runs during CI validates the compact webhook JSON files are parsed into a form that exactly matches the full unoptimized webhooks JSON.
The script update-webhooks.ts downloads the full webhook JSON from the GitHub REST API description repository:
import schemaImport from "rest-api-description/descriptions/api.github.com/dereferenced/api.github.com.deref.json";8fbd093 to
b541ddf
Compare
Overview
This PR optimizes the webhook payload JSON files used for GitHub Actions autocompletion and hover documentation, achieving a 55% size reduction for these files:
Changes
New Optimizations
Compact array format — Params converted from verbose objects to type-dispatched arrays:
Object deduplication — Shared structures extracted to
objects.jsonand referenced by indexString interning — 414 duplicate property names stored in
strings.jsonand referenced by indexSize Reduction
Testing
webhooks.full.json) to ensure no data lossDocumentation
Updated
docs/json-data-files.mdwith: