Skip to content

Saving to file lemmer results #7

@ivgenyk

Description

@ivgenyk

Hi,
I used lemmer to lemmatize a vector of 20K words and then wanted to save the results to a file. I receive an error saying "Error: EMFILE: too many open files, open 'c:\lemmered.txt'". When I am saving the original tokens vector (same number of words). It seems that it is a problem with the Lemmer.lemmatize function.

Do you know what can be the problem?
I tried many different ways of rewriting this and it seems that there is some part of the lemmer that afflicts the ability to write a file even after it has finished.

var Lemmer = require('lemmer');
const fs = require('fs');
let textFilePath = "C:/Qanta/leadgenPilot/OpenWhisk/sandBox/barc_text[9047].txt";
readFileIK(textFilePath) // this uses readFile as a promise resolving with the data
    .then(data => {
        let tokens = data
            // .substring(1,10000)
            .toLowerCase().split(/\W/g);
        Lemmer.lemmatize(tokens, function (err, wordsArray) {
            if (err) {
                console.log(err)
                throw (err)
            } else {
                let a = wordsArray.toString();
                console.log("lemmered ", wordsArray.length, "now will write to file")
                 fs.writeFile("lemmered.txt", a, (err) => {
                     if (err) {
                         console.log("error writing file ", err)
                     } else {
                        console.log("GREAT SUCCESS")
                    }
                 })
            }
        })
    })
function readFileIK(textFilePath) {
    return new Promise(function (resolve, reject) {
        fs.readFile(textFilePath, 'utf8', function (err, data) {
            if (err)
                reject(err)
            resolve(data)
        })
    })
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions