Skip to content

Add cache strategies with eviction policy#7

Open
DanielM08 wants to merge 6 commits intomasterfrom
addCacheStrategiesWithEvictionPolicy
Open

Add cache strategies with eviction policy#7
DanielM08 wants to merge 6 commits intomasterfrom
addCacheStrategiesWithEvictionPolicy

Conversation

@DanielM08
Copy link
Copy Markdown

@DanielM08 DanielM08 commented Jul 13, 2025

Here's a comprehensive pull request message describing the new changes:


🚀 Add Configurable Cache Strategies with Eviction Policies

📋 Summary

This PR introduces a major enhancement to the Remembered library by adding configurable cache strategies with eviction policies, allowing developers to manage memory usage more effectively and choose the most appropriate caching strategy for their use cases.

✨ New Features

🎯 Cache Eviction Policies

  • LRU (Least Recently Used): Removes least recently accessed items when capacity is reached
  • MRU (Most Recently Used): Removes most recently accessed items when capacity is reached
  • FIFO (First In, First Out): Removes oldest items when capacity is reached
  • Simple: No eviction policy, stores items indefinitely (default behavior)

⚙️ Enhanced Configuration

interface RememberedConfig<TResponse = unknown, TKey = string> {
  ttl: number | TtlFunction<TResponse, TKey>;
  evictionPolicy?: 'LRU' | 'MRU' | 'FIFO';
  capacity?: number; // Required when using eviction policies
  nonBlocking?: boolean;
  onReused?: (key: string) => void;
}

🔧 Cache Factory

  • New createCache factory function that instantiates appropriate cache implementation based on configuration
  • Automatic fallback to Simple cache for invalid or missing eviction policies
  • Type-safe cache creation with proper TypeScript support

🏗️ Architecture Changes

Refactored Cache Structure

  • Extracted LinkedList: Moved linked list operations from BaseCache to dedicated LinkedList class
  • Improved Separation of Concerns: Each cache implementation now focuses on its specific eviction logic
  • Enhanced Testability: Better isolation of components for more comprehensive testing

Cache Implementations

  • LruCache: Implements least recently used eviction strategy
  • MruCache: Implements most recently used eviction strategy
  • FifoCache: Implements first in, first out eviction strategy
  • SimpleCache: No eviction policy (original behavior)

🧪 Testing Enhancements

Comprehensive Test Coverage

  • Cache Strategy Tests: Full coverage for all eviction policies
  • Edge Cases: Capacity 0, invalid configurations, circular reference handling
  • Factory Tests: Complete testing of createCache function
  • Integration Tests: End-to-end testing of Remembered class with different strategies

Test Infrastructure

  • Custom Jest Serializer: Resolves circular reference issues in linked list tests
  • Helper Functions: Safe node inspection utilities for testing
  • Edge Case Coverage: Zero capacity, eviction scenarios, callback testing

📚 Documentation Updates

Enhanced README

  • Cache Strategy Guide: Detailed explanation of each eviction policy
  • Configuration Examples: Practical usage examples for all strategies
  • Use Case Recommendations: When to use each cache strategy
  • Advanced Configuration: Dynamic TTL functions and non-blocking mode

Updated TypeDoc

  • Constructor Documentation: Comprehensive parameter descriptions
  • Configuration Examples: Multiple usage patterns and scenarios
  • Type Safety: Complete TypeScript interface documentation

Breaking Changes

  • None: All changes are backward compatible
  • Default Behavior: Existing code continues to work with Simple cache strategy
  • Optional Features: New cache strategies are opt-in via configuration

Use Cases

LRU Cache

const remembered = new Remembered({
  ttl: 5000,
  evictionPolicy: 'LRU',
  capacity: 100
});

Best for: Frequently accessed data, user sessions, general purpose caching

MRU Cache

const remembered = new Remembered({
  ttl: 3000,
  evictionPolicy: 'MRU',
  capacity: 50
});

Best for: Data that becomes stale quickly, temporary data, rate limiting

FIFO Cache

const remembered = new Remembered({
  ttl: 10000,
  evictionPolicy: 'FIFO',
  capacity: 200
});

Best for: Time-sensitive data, logs, time-series data

🚀 Performance Benefits

  • Memory Management: Prevents unbounded memory growth with capacity limits
  • Efficient Eviction: O(1) operations for all eviction strategies
  • Flexible Configuration: Choose optimal strategy for specific use cases
  • Background Updates: Non-blocking mode for improved responsiveness

Files Changed

  • Core: src/cache-strategy/, src/remembered-config.ts, src/remembered.ts
  • Tests: test/unit/cache-strategy/, test/unit/index.spec.ts
  • Documentation: README.md, docs/classes/remembered.md
  • Configuration: package.json (test scripts)

🔍 Testing

npm test                    # Run all tests
npm run test:coverage      # Check test coverage
npm run test:only          # Run focused tests

All tests pass with comprehensive coverage of new functionality and existing features.


Ready for review! 🎉

private map = new Map<TKey, Promise<TResponse>>();
private nonBlockingMap = new Map<TKey, TResponse>();
private cache;
private nonBlockingCache;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you strongly define the types of these variables here?

private map = new Map<TKey, Promise<TResponse>>();
private nonBlockingMap = new Map<TKey, TResponse>();
private cache;
private nonBlockingCache;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I think nonBlockingMap must keep being a simple Map. The idea behind it is to always remember the last value of the given key, no matter if it has expired or not, so I must not be influenced by eviction rules, so I'd recommend to keep this property as it was

this.cache.delete(key);
} else if (result !== Empty) {
this.pacer?.schedulePurge(key, ttl, result);
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at all the changes you've made here, you can remove almost all of them by keeping the current type and name of the variables, and also preventing breaking changes in a more rigid manner

private removeImmediately: boolean;
private onReused?: (...args: any[]) => void;

constructor(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to throw an error in the constructor if someone tries to use nonBlocking = true and an eviction rule in the configs? It looks to me like exclusive concepts that should never be used together

@@ -0,0 +1,5 @@
export interface Cache<TResponse = unknown, TKey = string> {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's better to keep the idiomatic around the Map word, instead of cache, to prevent confusion with dedicated cache services.
You can call it, for example, EvictionableMap. I also recommend you to invert the class generic parameters, to make it's contract compatible with Map<TKey, TResponse | Promise>

@@ -0,0 +1,26 @@
import { Cache } from '../core/cache';

export class SimpleCache<TResponse = unknown, TKey = string>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class can be completely avoided if you invert the interface generic parameters, as you can just use Map instead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants