Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- **cache**: A cache backend that fails to load (e.g. `better-sqlite3` ABI mismatch after a Node major upgrade, `ERR_DLOPEN_FAILED`) is no longer misclassified as database corruption. Previously the constructor's catch-all renamed the user's healthy `cache.db` to `cache.db.corrupt-<timestamp>` and recreated an empty database — verified to quarantine a 2,646-entry cache that passed `integrity_check`. Native-module load failures now leave the database and its `-wal`/`-shm` sidecars untouched; genuine corruption still triggers the rename-aside recovery. `deepl translate` and `deepl write` degrade to running without a cache (single warning per process, exit code 0) instead of crashing, since the cache backend is loaded lazily behind a warn-once latch; `deepl cache …` subcommands, which cannot run cacheless, fail with an actionable error suggesting a reinstall or matching Node version.
- **sync**: `deepl sync --frozen` now reports an accurate key count when drift is caused by a newly-added target locale. Previously the message read `Sync drift detected: 0 new, 0 stale keys.` because the frozen branch in `processBucket` short-circuited before promoting current-status keys missing a target-locale translation into `newKeys` — even though `--dry-run` against the same state correctly reported the backfill count. The drift exit code (10) is unchanged. The drift message now also surfaces `deletedKeys` and only mentions nonzero categories, mirroring the success-path summary format.

### Security
Expand Down
6 changes: 5 additions & 1 deletion docs/TROUBLESHOOTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,16 +417,20 @@ rm ~/.cache/deepl-cli/cache.db # or ~/.deepl-cli/cache.db for legacy installat
deepl cache enable
```

### "Cache database corrupted" with NODE_MODULE_VERSION mismatch
### "Translation cache backend failed to load" with NODE_MODULE_VERSION mismatch

**Cause:** The `better-sqlite3` native addon was compiled for a different Node.js version than the one currently running. This happens when you switch Node.js versions (e.g., via `nvm use`, `fnm`, or a system upgrade) without rebuilding native modules.

Translation and write commands keep working with caching disabled for the run; your cache database is not modified. `deepl cache` subcommands fail until the backend loads again.

**Solution:**

```bash
npm rebuild better-sqlite3
```

or reinstall the CLI / switch back to the Node.js version it was installed with.

---

## Document Translation Issues
Expand Down
47 changes: 47 additions & 0 deletions src/cli/cache-loader.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/**
* Lazy cache-service loader for the CLI entry point.
* The cache backend (a native module today, node:sqlite tomorrow) can
* fail to load on runtimes it wasn't built for. Commands that merely
* benefit from a cache must degrade to running without one instead of
* crashing, with a single warning per process.
*/

import type { CacheService, CacheServiceOptions } from '../storage/cache.js';
import { Logger } from '../utils/logger.js';
import { errorMessage } from '../utils/error-message.js';
import { isNativeModuleLoadError } from '../utils/native-module-error.js';

type CacheModule = Pick<typeof import('../storage/cache.js'), 'CacheService'>;

export function createCacheServiceGetter(
getOptions: () => CacheServiceOptions,
importCacheModule: () => Promise<CacheModule> = () => import('../storage/cache.js'),
): () => Promise<CacheService | undefined> {
let instance: CacheService | undefined;
let unavailable = false;

return async (): Promise<CacheService | undefined> => {
if (instance || unavailable) {
return instance;
}
try {
const { CacheService: CacheSvc } = await importCacheModule();
instance = CacheSvc.getInstance(getOptions());
} catch (error) {
unavailable = true;
const detail = errorMessage(error);
if (isNativeModuleLoadError(error)) {
Logger.warn(
`Translation cache backend failed to load (${detail}). ` +
'Your cache database has not been modified. Caching is disabled for this run. ' +
'Reinstall the CLI, or run it with the Node.js version it was installed with, to restore caching.',
);
} else {
Logger.warn(
`Translation cache is unavailable (${detail}). Caching is disabled for this run.`,
);
}
}
return instance;
};
}
24 changes: 18 additions & 6 deletions src/cli/commands/register-cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,29 @@ import chalk from 'chalk';
import type { ConfigService } from '../../storage/config.js';
import type { CacheService } from '../../storage/cache.js';
import { Logger } from '../../utils/logger.js';
import { ConfigError } from '../../utils/errors.js';

export function registerCache(
program: Command,
deps: {
getConfigService: () => ConfigService;
getCacheService: () => Promise<CacheService>;
getCacheService: () => Promise<CacheService | undefined>;
handleError: (error: unknown) => never;
},
): void {
const { getConfigService, getCacheService, handleError } = deps;

async function requireCacheService(): Promise<CacheService> {
const cacheService = await getCacheService();
if (!cacheService) {
throw new ConfigError(
'Cache backend is unavailable, so cache commands cannot run.',
'Reinstall the CLI, or run it with the Node.js version it was installed with.',
);
}
return cacheService;
}

program
.command('cache')
.description('Manage translation cache')
Expand All @@ -32,7 +44,7 @@ Examples:
.action(async (options: { format?: string }) => {
try {
const { CacheCommand } = await import('./cache.js');
const cacheCommand = new CacheCommand(await getCacheService(), getConfigService());
const cacheCommand = new CacheCommand(await requireCacheService(), getConfigService());
const stats = await cacheCommand.stats();
if (options.format === 'json') {
Logger.output(JSON.stringify(stats, null, 2));
Expand Down Expand Up @@ -61,7 +73,7 @@ Examples:
try {
if (options.dryRun) {
const { CacheCommand } = await import('./cache.js');
const cacheCommand = new CacheCommand(await getCacheService(), getConfigService());
const cacheCommand = new CacheCommand(await requireCacheService(), getConfigService());
const stats = await cacheCommand.stats();
const totalSizeMB = (stats.totalSize / (1024 * 1024)).toFixed(2);
const lines = [
Expand All @@ -83,7 +95,7 @@ Examples:
}

const { CacheCommand } = await import('./cache.js');
const cacheCommand = new CacheCommand(await getCacheService(), getConfigService());
const cacheCommand = new CacheCommand(await requireCacheService(), getConfigService());
await cacheCommand.clear();
Logger.success(chalk.green('\u2713 Cache cleared successfully'));
} catch (error) {
Expand All @@ -106,7 +118,7 @@ Examples:
}

const { CacheCommand } = await import('./cache.js');
const cacheCommand = new CacheCommand(await getCacheService(), getConfigService());
const cacheCommand = new CacheCommand(await requireCacheService(), getConfigService());
await cacheCommand.enable(maxSizeBytes);
Logger.success(chalk.green('\u2713 Cache enabled'));

Expand All @@ -126,7 +138,7 @@ Examples:
.action(async () => {
try {
const { CacheCommand } = await import('./cache.js');
const cacheCommand = new CacheCommand(await getCacheService(), getConfigService());
const cacheCommand = new CacheCommand(await requireCacheService(), getConfigService());
await cacheCommand.disable();
Logger.success(chalk.green('\u2713 Cache disabled'));
} catch (error) {
Expand Down
2 changes: 1 addition & 1 deletion src/cli/commands/service-factory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ export interface ServiceDeps {
createDeepLClient: CreateDeepLClient;
getApiKeyAndOptions: GetApiKeyAndOptions;
getConfigService: () => ConfigService;
getCacheService: () => Promise<CacheService>;
getCacheService: () => Promise<CacheService | undefined>;
handleError: (error: unknown) => never;
}

Expand Down
29 changes: 12 additions & 17 deletions src/cli/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import { readFileSync } from 'fs';
import { fileURLToPath } from 'url';
import { dirname, join, resolve, isAbsolute, extname } from 'path';
import { ConfigService } from '../storage/config.js';
import type { CacheService } from '../storage/cache.js';
import { createCacheServiceGetter } from './cache-loader.js';
import { resolvePaths } from '../utils/paths.js';
import type { DeepLClient } from '../api/deepl-client.js';
import { Logger } from '../utils/logger.js';
Expand Down Expand Up @@ -57,22 +57,17 @@ const paths = resolvePaths();

// Create config service - can be overridden by --config flag
let configService = new ConfigService(paths.configFile);
let cacheService: CacheService | null = null;

async function getCacheService(): Promise<CacheService> {
if (!cacheService) {
const { CacheService: CacheSvc } = await import('../storage/cache.js');
const configTtl = configService.getValue<number>('cache.ttl');
const configMaxSize = configService.getValue<number>('cache.maxSize');
cacheService = CacheSvc.getInstance({
dbPath: paths.cacheFile,
// Config TTL is in seconds, CacheService expects milliseconds
ttl: configTtl !== undefined ? configTtl * 1000 : undefined,
maxSize: configMaxSize,
});
}
return cacheService;
}

const getCacheService = createCacheServiceGetter(() => {
const configTtl = configService.getValue<number>('cache.ttl');
const configMaxSize = configService.getValue<number>('cache.maxSize');
return {
dbPath: paths.cacheFile,
// Config TTL is in seconds, CacheService expects milliseconds
ttl: configTtl !== undefined ? configTtl * 1000 : undefined,
maxSize: configMaxSize,
};
});

/**
* Handle error and exit with appropriate exit code
Expand Down
16 changes: 9 additions & 7 deletions src/services/translation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import * as crypto from 'crypto';
import { DeepLClient, TranslationResult, isTranslationResult, UsageInfo, LanguageInfo } from '../api/deepl-client.js';
import { ConfigService } from '../storage/config.js';
import { CacheService } from '../storage/cache.js';
import type { CacheService } from '../storage/cache.js';
import { TranslationOptions, Language, TranslationMemory } from '../types/index.js';
import { Logger } from '../utils/logger.js';
import { mapWithConcurrency, MULTI_TARGET_CONCURRENCY } from '../utils/concurrency.js';
Expand Down Expand Up @@ -40,14 +40,16 @@ export const TRANSLATE_BATCH_SIZE = 50; // DeepL API max texts per request
export class TranslationService {
private client: DeepLClient;
private config: ConfigService;
private cache: CacheService;
// No cache means "run cacheless" — the CLI passes undefined when the
// cache backend is unavailable (see cli/cache-loader.ts).
private cache?: CacheService;
private languageCache: Map<'source' | 'target', { data: LanguageInfo[]; timestamp: number }> = new Map();
private readonly LANGUAGE_CACHE_TTL = 24 * 60 * 60 * 1000; // 24 hours

constructor(client: DeepLClient, config: ConfigService, cache?: CacheService) {
this.client = client;
this.config = config;
this.cache = cache ?? CacheService.getInstance();
this.cache = cache;
}

/**
Expand Down Expand Up @@ -112,7 +114,7 @@ export class TranslationService {
const cacheKey = this.generateCacheKey(processedText, translationOptions);

if (shouldUseCache) {
const cachedResult = this.cache.get(cacheKey, isTranslationResult);
const cachedResult = this.cache?.get(cacheKey, isTranslationResult);
if (cachedResult) {
Logger.verbose('[verbose] Cache hit');
return {
Expand All @@ -131,7 +133,7 @@ export class TranslationService {

// Store in cache
if (shouldUseCache) {
this.cache.set(cacheKey, result);
this.cache?.set(cacheKey, result);
}

return {
Expand Down Expand Up @@ -198,7 +200,7 @@ export class TranslationService {
const cacheKey = this.generateCacheKey(text, translationOptions);

if (cacheEnabled) {
const cachedResult = this.cache.get(cacheKey, isTranslationResult);
const cachedResult = this.cache?.get(cacheKey, isTranslationResult);
if (cachedResult) {
results[i] = cachedResult;
continue;
Expand Down Expand Up @@ -281,7 +283,7 @@ export class TranslationService {
// Cache the result (only once per unique text)
if (cacheEnabled) {
const cacheKey = this.generateCacheKey(text, translationOptions);
this.cache.set(cacheKey, result);
this.cache?.set(cacheKey, result);
}
}
}
Expand Down
12 changes: 7 additions & 5 deletions src/services/write.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import * as crypto from 'crypto';
import { DeepLClient } from '../api/deepl-client.js';
import { ConfigService } from '../storage/config.js';
import { CacheService } from '../storage/cache.js';
import type { CacheService } from '../storage/cache.js';
import { WriteOptions, WriteImprovement, isWriteImprovementArray } from '../types/index.js';
import { Logger } from '../utils/logger.js';
import { ValidationError, ConfigError } from '../utils/errors.js';
Expand All @@ -18,7 +18,9 @@ export interface WriteServiceOptions {
export class WriteService {
private client: DeepLClient;
private config: ConfigService;
private cache: CacheService;
// No cache means "run cacheless" — the CLI passes undefined when the
// cache backend is unavailable (see cli/cache-loader.ts).
private cache?: CacheService;

constructor(client: DeepLClient, config: ConfigService, cache?: CacheService) {
if (!client) {
Expand All @@ -31,7 +33,7 @@ export class WriteService {

this.client = client;
this.config = config;
this.cache = cache ?? CacheService.getInstance();
this.cache = cache;
}

/**
Expand Down Expand Up @@ -62,7 +64,7 @@ export class WriteService {
const cacheKey = this.generateCacheKey(text, options);

if (shouldUseCache) {
const cachedResult = this.cache.get(cacheKey, isWriteImprovementArray);
const cachedResult = this.cache?.get(cacheKey, isWriteImprovementArray);
if (cachedResult) {
Logger.verbose('[verbose] Cache hit');
return cachedResult;
Expand All @@ -73,7 +75,7 @@ export class WriteService {
const improvements = await this.client.improveText(text, options);

if (shouldUseCache) {
this.cache.set(cacheKey, improvements);
this.cache?.set(cacheKey, improvements);
}

return improvements;
Expand Down
7 changes: 7 additions & 0 deletions src/storage/cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { resolvePaths } from '../utils/paths.js';
import { ConfigError } from '../utils/errors.js';
import { Logger } from '../utils/logger.js';
import { errorMessage } from '../utils/error-message.js';
import { isNativeModuleLoadError } from '../utils/native-module-error.js';

export interface CacheServiceOptions {
dbPath?: string;
Expand Down Expand Up @@ -69,6 +70,12 @@ export class CacheService {
try {
this.openDatabase(dbPath);
} catch (error) {
// A backend that cannot load (ABI mismatch after a Node upgrade,
// missing binding) is not corruption: the database on disk is
// healthy, so renaming it aside would throw away a warm cache.
if (isNativeModuleLoadError(error)) {
throw error;
}
// Corrupted DB: rename-aside rather than unlink, so the user keeps
// 30 days of cache history (and a forensic artifact) instead of
// losing both silently. Suffix with a timestamp so repeated
Expand Down
24 changes: 24 additions & 0 deletions src/utils/native-module-error.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/**
* Classifies errors thrown while loading the cache's storage backend.
* These indicate the module itself cannot load (ABI mismatch after a
* Node upgrade, missing binding, runtime without node:sqlite) — the
* database file on disk is healthy and must not be touched.
*/

const NATIVE_LOAD_ERROR_CODES = new Set([
'ERR_DLOPEN_FAILED',
'ERR_UNKNOWN_BUILTIN_MODULE',
'MODULE_NOT_FOUND',
'ERR_MODULE_NOT_FOUND',
]);

export function isNativeModuleLoadError(error: unknown): boolean {
if (!(error instanceof Error)) {
return false;
}
const code = (error as NodeJS.ErrnoException).code;
if (code && NATIVE_LOAD_ERROR_CODES.has(code)) {
return true;
}
return error.message.includes('NODE_MODULE_VERSION');
}
Loading