Repository navigation
fix(run-with-apify-token): support all config modes - #219
JuanGalilea wants to merge 7 commits into
Conversation
|
this will conflict with #225, the conflict is minor thought since they are somewhat orthogonal |
metalwarrior665
left a comment
There was a problem hiding this comment.
Reusing the config parser is the right fix for grouped mode. I rebuilt the bundles locally (they match what's committed) and the bundle and config tests pass.
This depends on the rebase requested on #213: once #213 ports #225, this PR's test needs WORKFLOW_SECRETS (inline). The other comments are small.
FYI, the generated script grows from about 3 KB to 467 KB, almost all of it zod. It's marked generated and the size doesn't matter at runtime, so no change needed.
Generated by Claude Code
| actorConfigs = await readBaseConfigFile(); | ||
| } catch (error) { | ||
| fail(`Cannot read "${configPath}": ${error instanceof Error ? error.message : String(error)}`); | ||
| fail(`Cannot read "${CONFIG_FILE_NAME}": ${error instanceof Error ? error.message : String(error)}`); |
There was a problem hiding this comment.
This prefix is now wrong in most cases, and it repeats the message underneath:
- Missing file:
Cannot read "apify-test-tools.config.json": Config file "apify-test-tools.config.json" not found in the current directory… - Schema error:
Cannot read "apify-test-tools.config.json": ✖ Invalid string: must match pattern …, even though the file was read fine.
readConfigFileContents and the parsers already write messages for the user, so passing error.message through, or using a neutral prefix like Invalid config:, would read better.
Generated by Claude Code
There was a problem hiding this comment.
solved, got rid of the whole prefix
| `Building those Actors will fail.`, | ||
| ); | ||
| } | ||
| const tokenNames = [...new Set(actorConfigs.map(({ tokenEnvVar }) => tokenEnvVar))].sort(); |
There was a problem hiding this comment.
The schemas accept "tokenEnvVar": "" (z.string() in legacy.ts and grouped.ts). The old script filtered empty names out, but now "" lands in tokenNames, and the run prints:
Warning: apify-test-tools.config.json declares , but no such secret was passed to this workflow. …
.min(1) on those schemas would fix it here and in the CLI. Filtering '' here is the local alternative.
Also worth a line in the description: an entry without tokenEnvVar used to be a warning (the entriesWithoutToken path). Now it fails the whole step at parse time. That's consistent with the CLI, which rejects the same config anyway, but users will notice.
Generated by Claude Code
There was a problem hiding this comment.
set all parsers to enforce min length 1
|
|
||
| /** Stage 1 — the config file as a plain object, or a failure phrased for whoever wrote it. */ | ||
| const readConfigFileContents = async (): Promise<Record<string, unknown>> => { | ||
| export const readConfigFileContents = async (): Promise<Record<string, unknown>> => { |
There was a problem hiding this comment.
Nit: this export isn't needed. readConfigFileContents is only used inside this file (by readConfigFile and readBaseConfigFile), and check-unused will flag it once that check is turned on.
Generated by Claude Code
There was a problem hiding this comment.
got rid of export
| }) satisfies PathlessConfig, | ||
| // `satisfies` so its not shown as `PathlessConfig` type on output or IDE, but the expanded form |
There was a problem hiding this comment.
Nit: this comment sits in .map()'s argument list, after the callback, so it reads like a second argument. Moving it above the object literal (or the return) would make it clearer.
Generated by Claude Code
There was a problem hiding this comment.
moved it to right before of the return of the arrow function
| // TODO: drop folder uniqueness (this requires several changes on other places) | ||
| if (seenFolders.has(entry.folder.path)) { | ||
| errors.push(`Duplicate folder "${body[index].folder}". Each actor must have a unique folder.`); | ||
| errors.push(`Duplicate folder "${resolved[index].folder}". Each actor must have a unique folder.`); |
There was a problem hiding this comment.
resolved[index] assumes body and resolved are in the same order. That holds because initializePaths maps one-to-one, but it's not obvious from the signature. A one-line comment (or doc on the resolved parameter) would help.
Generated by Claude Code
There was a problem hiding this comment.
got rid of resolved altogether since it was only being used to print the raw folder.
It could cause some confusion but i guess thats what you get for passing 2 paths that normalize to the same path. In theory it should never happen, why would you pass in notIncluded/../path/to/./file doesn't even make sense.
| for (const name of ['one', 'two']) { | ||
| await mkdir(path.join(repoDir, 'actors', name), { recursive: true }); | ||
| } |
There was a problem hiding this comment.
After b23803d the script no longer checks folders, so these directories are leftover setup. Removing them, or pointing the config at folders that don't exist, would also test the "no path checks" behaviour end to end. Right now only the unit test covers it.
Generated by Claude Code
| cwd: repoDir, | ||
| env: { | ||
| PATH: process.env.PATH ?? '', | ||
| ALL_SECRETS: JSON.stringify({ |
There was a problem hiding this comment.
This will need to become WORKFLOW_SECRETS once #213 is rebased onto #225 (see my comment there). While you're here, a case where a token comes only from inside CONFIG_SECRETS would make sure the next rebase can't drop that path unnoticed.
Generated by Claude Code
b23803d to
308350c
Compare
As mentioned in #213, script currently takes only the default (legacy) config mode.
With it being moved to the typescript source and built from it, we can now reuse our config parsing code to get all tokens without having to care about config modes or re-writing the logic by hand.
This reuses the shared config parser in
run-with-apify-tokensto support both legacy and grouped configs, including group defaults and actor overrides.Only read the config settings needed for token forwarding, without checking local paths or loading
actor.json.Empty or missing required tokenEnvVar values now fail config validation instead of producing warnings. Declared tokens without a matching secret still produce warnings.