Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 6 additions & 14 deletions .github/workflows/golang.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,12 @@ on:
default: true
type: boolean
description: False to skip code-scan exec
vulterability-scan:
vulnerability-scan:
required: false
default: true
type: boolean
description: False to skip vulterability-scan exec
vulterability-scan-skip-dirs:
description: False to skip vulnerability-scan exec
vulnerability-scan-skip-dirs:
required: false
default: "vendor,.github"
type: string
Expand Down Expand Up @@ -189,14 +189,6 @@ jobs:
- name: Test
run: |
make ci-test
- name: Login to GitHub Container Registry
uses: docker/login-action@v4
if: ${{ inputs.code-scan }}
# we log in the registry to code check also images
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: SonarCloud Code Scan
if: ${{ inputs.code-scan }}
uses: SonarSource/sonarqube-scan-action@v7.1.0
Expand Down Expand Up @@ -267,7 +259,7 @@ jobs:
retention-days: 400
vulnerability:
needs: changes
if: ${{ inputs.vulterability-scan}}
if: ${{ inputs.vulnerability-scan}}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
Expand All @@ -277,7 +269,7 @@ jobs:
with:
scan-type: fs
scan-ref: .
skip-dirs: ${{ inputs.vulterability-scan-skip-dirs }}
skip-dirs: ${{ inputs.vulnerability-scan-skip-dirs }}
scanners: "vuln,secret,misconfig"
hide-progress: true
output: trivy.txt
Expand All @@ -302,7 +294,7 @@ jobs:
with:
scan-type: fs
scan-ref: .
skip-dirs: ${{ inputs.vulterability-scan-skip-dirs }}
skip-dirs: ${{ inputs.vulnerability-scan-skip-dirs }}
scanners: "vuln,secret,misconfig"
exit-code: "1"
severity: MEDIUM,HIGH,CRITICAL
Expand Down
232 changes: 232 additions & 0 deletions .github/workflows/js.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,232 @@
# Copyright 2024 Zaphiro Technologies
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: JS Lint & Test

concurrency:
group: ${{github.repository}}-${{ github.ref_name }}-js
cancel-in-progress: true

# permissions:
# id-token: write # This is required for requesting the JWT, which is needed to authenticate with AWS and pull DVC datasets

on:
workflow_call:
inputs:
code-scan:
required: false
default: true
type: boolean
description: False to skip code-scan exec
vulnerability-scan:
required: false
default: true
type: boolean
description: False to skip vulnerability-scan exec
vulnerability-scan-skip-dirs:
required: false
default: "vendor,.github"
type: string
description: Comma separated list of directories where vulnerability check is skipped
pull-dvc-datasets:
required: false
default: false
type: boolean
description: True to install and configure dvc
repositories:
required: false
type: string
default: ""
description: Comma or newline-separated list of repositories to grant access to during the JS build.
jobs:
changes:
runs-on: ubuntu-latest
outputs:
files_changed: ${{ steps.filter.outputs.js == 'true' || steps.filter.outputs.scripts == 'true' }}
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 2 # To retrieve the preceding commit.
- uses: dorny/paths-filter@v4
id: filter
with:
list-files: shell
filters: |
scripts:
- 'Makefile'
- '.github/workflows/js.*'
- '**/*.sql'
- '.docker/*.yml'
- '.docker/*.yaml'
js:
- '**/*.ts'
- '**/*.js'
- '**/*.tsx'
- '**/*.jsx'
- 'package.json'
lint:
needs: changes
runs-on: ubuntu-latest
permissions:
contents: write
if: ${{ needs.changes.outputs.files_changed == 'true' && github.event.pull_request && github.event.pull_request.head.repo.full_name == github.repository }}
steps:
- uses: actions/create-github-app-token@v3
id: generate-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.APP_SECRET }}
permission-contents: write
Comment thread
chicco785 marked this conversation as resolved.
- uses: actions/checkout@v6
with:
ref: ${{github.event.pull_request.head.ref}} # Can't commit on detached PR merge commit, so this checkouts the branch
token: ${{ steps.generate-token.outputs.token }}
- run: git config --global "url.https://x-access-token:${{ steps.generate-token.outputs.token }}@github.com/.insteadOf" "https://github.com/"
- uses: actions/setup-node@v6
with:
node-version-file: package.json
- name: "Enable Corepack"
run: corepack enable
- name: "Install packages"
run: |
YARN_ENABLE_SCRIPTS=0 yarn install
- name: Check types
run: yarn typecheck
- name: Lint
run: |
yarn fix
- name: Commit lint changes
run: |
git config --global user.name 'Bot'
git config --global user.email 'bot@zaphiro.ch'
git commit -am "Automated lint fixes [dependabot skip]" || echo "No changes to commit"
git push
test:
needs: changes
runs-on: ubuntu-latest
if: ${{ needs.changes.outputs.files_changed == 'true' }}
steps:
- uses: actions/create-github-app-token@v3
id: generate-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.APP_SECRET }}
repositories: ${{ github.event.repository.name }}${{ inputs.repositories && format(',{0}', inputs.repositories) || '' }}
permission-contents: read
- uses: actions/checkout@v6
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: Configure AWS Credentials
if: ${{ inputs.pull-dvc-datasets }}
id: creds
uses: aws-actions/configure-aws-credentials@v6
with:
aws-region: eu-south-1
role-to-assume: arn:aws:iam::058264335177:role/zaphiro-github-role-demo
output-credentials: true
- name: Install DVC
if: ${{ inputs.pull-dvc-datasets }}
run: |
pipx install 'dvc[s3]==3.67.0'
- name: Set-up git credentials for dvc
if: ${{ inputs.pull-dvc-datasets }}
env:
GH_TOKEN: ${{ steps.generate-token.outputs.token }}
run: |
gh auth setup-git
git config --global credential.helper '!/usr/bin/gh auth git-credential'
- name: Pull DVC Data
env:
GH_TOKEN: ${{ steps.generate-token.outputs.token }}
if: ${{ inputs.pull-dvc-datasets }}
run: dvc pull -v --allow-missing
- run: git config --global "url.https://x-access-token:${{ steps.generate-token.outputs.token }}@github.com/.insteadOf" "https://github.com/"
- uses: actions/setup-node@v6
with:
node-version-file: package.json
- name: "Enable Corepack"
run: corepack enable
- name: "Install packages"
run: |
YARN_ENABLE_SCRIPTS=0 yarn install
- name: Prepare build
run: |
make ci-pre-build
- name: Login to GitHub Container Registry
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Unit tests
run: make test
- name: Build
run: |
yarn build
- name: Upload build artifact
uses: actions/upload-pages-artifact@v5
with:
path: ./build
- name: SonarCloud Code Scan
if: ${{ inputs.code-scan }}
uses: SonarSource/sonarqube-scan-action@v7.1.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
vulnerability:
needs: changes
if: ${{ inputs.vulnerability-scan}}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Run Trivy vulnerability scanner (for summary)
# trivy-action v0.35.0 (safe version)
uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1
with:
scan-type: fs
scan-ref: .
skip-dirs: ${{ inputs.vulnerability-scan-skip-dirs }}
scanners: "vuln,secret,misconfig"
hide-progress: true
output: trivy.txt
env:
TRIVY_SKIP_DB_UPDATE: true
TRIVY_SKIP_JAVA_DB_UPDATE: true
- name: Publish Trivy Output to Summary
run: |
if [[ -s trivy.txt ]]; then
{
echo "### Trivy Security Output"
echo ""
echo '```text'
cat trivy.txt
echo '```'
} >> $GITHUB_STEP_SUMMARY
fi
# this comes as last or scan results won't be uploaded
- name: Run Trivy vulnerability scanner (security treshold)
# trivy-action v0.35.0 (safe version)
uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1
with:
scan-type: fs
scan-ref: .
skip-dirs: ${{ inputs.vulnerability-scan-skip-dirs }}
scanners: "vuln,secret,misconfig"
exit-code: "1"
severity: MEDIUM,HIGH,CRITICAL
# On a subsequent call to the action we know trivy is already installed so can skip this
skip-setup-trivy: true
env:
TRIVY_SKIP_DB_UPDATE: true
TRIVY_SKIP_JAVA_DB_UPDATE: true
14 changes: 7 additions & 7 deletions .github/workflows/python.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,16 @@ on:
default: true
type: boolean
description: False to skip code-scan exec
vulterability-scan:
vulnerability-scan:
required: false
default: true
type: boolean
description: False to skip vulterability-scan exec
vulterability-scan-skip-dirs:
description: False to skip vulnerability-scan exec
vulnerability-scan-skip-dirs:
required: false
default: "vendor,.github"
type: string
description: Comma separated list of directories where vulterability check is skipped
description: Comma separated list of directories where vulnerability check is skipped
pull-dvc-datasets:
required: false
default: false
Expand Down Expand Up @@ -212,7 +212,7 @@ jobs:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
vulnerability:
needs: changes
if: ${{ inputs.vulterability-scan}}
if: ${{ inputs.vulnerability-scan}}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
Expand All @@ -222,7 +222,7 @@ jobs:
with:
scan-type: fs
scan-ref: .
skip-dirs: ${{ inputs.vulterability-scan-skip-dirs }}
skip-dirs: ${{ inputs.vulnerability-scan-skip-dirs }}
scanners: "vuln,secret,misconfig"
hide-progress: true
output: trivy.txt
Expand All @@ -247,7 +247,7 @@ jobs:
with:
scan-type: fs
scan-ref: .
skip-dirs: ${{ inputs.vulterability-scan-skip-dirs }}
skip-dirs: ${{ inputs.vulnerability-scan-skip-dirs }}
scanners: "vuln,secret,misconfig"
exit-code: "1"
severity: MEDIUM,HIGH,CRITICAL
Expand Down
4 changes: 4 additions & 0 deletions README.MD
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ The repository includes:
Applications. The workflow includes authentication to GitHub Container
Registry in case tests rely on private images. The workflow also scans for
vulnerabilities.
- [`js`](.github/workflows/js.yaml): typecheck, lint, test and build JS/TS
applications. The workflow can auto-fix lint issues on same-repository pull
requests, optionally pull DVC datasets, upload the build artifact, and scan
for code quality and vulnerabilities.
- [`license`](.github/workflows/license.yaml): add licensing information in file
headers and check dependencies licensing compatibility.
- [`lint`](.github/workflows/markdown.yaml): lint all Markdown and Yaml files.
Expand Down
4 changes: 3 additions & 1 deletion RELEASE_NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

### Features

- js workflow: shared workflow for javascript builds based on yarn2 (PR #287 by
@chicco785)
- docker wf: support login to docker hub to retrieve dhi.io images (PR #286 by
@chicco785)
- python workflow: ensure python version consistency (PR #246 by @chicco785)
Expand Down Expand Up @@ -98,9 +100,9 @@

### Dependencies

- Bump actions/github-script from 8 to 9 (PR #283 by @dependabot[bot])
- Bump aquasecurity/trivy-action from 0.35.0 to 0.36.0 (PR #285 by
@dependabot[bot])
- Bump actions/github-script from 8 to 9 (PR #283 by @dependabot[bot])
- Bump oras-project/setup-oras from 1 to 2 (PR #280 by @dependabot[bot])
- Bump SonarSource/sonarqube-scan-action from 7.0.0 to 7.1.0 (PR #279 by
@dependabot[bot])
Expand Down
Loading