enhancement: consistent lazy loading - #11763
Conversation
Jetz72
added
the
CardDB
There was a problem hiding this comment.
this still breaks matrix decks: ba71e6a
it would be acceptable to just skip their loading like in that old commit for now, since it's probably better if we leave the user facing preference disabled at this point anyway 🤔
|
Hey @tool4ever thanks for the speedy review. I think the fix was a one liner in the last commit.
And indeed it seems to be skipping some decks? (I'm not sure I fully understand) 
If I understsand correctly, it's not so much that it was "broken", is that these decks naturally will lazily load the whole db, defeating the purpose of the flag. |
|
hmn I think they contain mostly their own serialized PaperCard, didn't seem to trigger any full load you should probably test with other features/modes a bit too so any other wrong assumptions in the code might get uncovered - isolated suite alone isn't enough for such a complex mechanism |
|
Yea I got confused on the folder cos I originally misunderstood the problem. |
|
sure no hurry, I doesn't have to work with everything as long as we understand the limits clearly |
|
Hey @tool4ever.
This is a log I added on my local build counting every 1000 lazily loaded cards: I think this is expected behaviour, since I imported quite a bit of decks from some of the archives. When a path needs After they all load it essentially becomes functionally equivalent to eager mode. After reading a bit I left out Adventure mode from the testing. I found one crash on the deck editor, but I believe it's pre-existing, as I could reproduce on eager mode on master as well: Please let me know if you'd like to test anything else. |
4 participants
The problem
A forge engine needs the full cardb in memory (about 400MB) to run a single game.
This is impractical for someone hosting a server, or in my case, for running forge in the browser via GraalVM and web workers.
The proposed fix
Re enable
LOAD_CARD_SCRIPTS_LAZILYand make lazy lookups consistent with eager lookup.Eager mode is unchanged, the following only applies when lazy mode is enabled.
At startup every card script is stream-parsed to build a
{name: file}index covering all names from all scripts.This includes CopyFaces, flavour names, double
//cards. The parsed rules are discarded to keep resident memory light. Builds in about 1s and runs inside a 64MB heap in my case.A lookup then becomes a matter of resolving a file from the name from the index. This should, as far as I understand, yield identical coverage to eager mode.
A minor util was included,
resetLazyLoadedCards(), which allows the caller to drop accumulated cards. Custom cards are re-fed afterwards so they survive the reset. I wasn't sure about these so just in case, I'm never clearing them up. I don't fully understand the impact this has on custom cards.ensureAllCardsLoaded
Theres a bunch of places in which I had to call
ensureAllCardsLoaded. I don't particularly like this, but theres some nasty cards like jhoira-of-the-ghitu-avatar that as far as I understand would need to take a random sorcery from the whole card pool. That would require the whole thing to be in memory. Theres other few cases that require this, but I think they are all warranted.Testing
Compiles against master (
forge-core,forge-game,forge-gui).We are also about to test this live on the manabrew app for stability.
The following cards, made up of edge cases, loads fine under lazy loading: