Added ; and , support in parse_turtle

[lucafabbian]

parse_turtle was not able to ingest things such as ex:Alex ex:Age 10; ex:Friend ex:Bob, as it was missing support for ; and , .

Now it should work.

Tests are not able to go through, but I don't think it's my fault, as I get the same error even without my code changes.

     Running tests/ml_predict_candle_runtime.rs (target/debug/deps/ml_predict_candle_runtime-f366963faf61c852)

running 10 tests
test binary_always_emit_with_probability_companion ... ok
test unused_output_variable_errors ... ok
test preserves_non_ml_conclusions ... ok
test multiple_conclusion_predicates_errors ... ok
test head_only_output_variable_materializes ... ok
test empty_input_rerun_clears_stale ... FAILED
test input_filter_preserved ... ok
test model_name_mismatch_errors ... ok
test missing_anchor_in_input_select_errors ... ok
test rerun_cleans_stale_predictions ... ok

failures:

---- empty_input_rerun_clears_stale stdout ----
Processing rule with ML.PREDICT
ML output variable: ?label

thread 'empty_input_rerun_clears_stale' panicked at kolibrie/tests/ml_predict_candle_runtime.rs:435:44:
second run failed: "Row 0: feature var ?x0 value \"\" is not numeric: Non-numeric RDF term in neural feature vector: "
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    empty_input_rerun_clears_stale

test result: FAILED. 9 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.11s

[ladroid]

@lucafabbian Thanks for the parse_turtle update. I tested the shorthand case:

ex:Alex ex:Age 10; ex:Friend ex:Bob, ex:Charlie .

The current implementation does not work unless tokenize_turtle_star_line also emits ; , and . as separate delimiter tokens outside IRIs/literals/quoted triples. With only the parse_turtle change, the object becomes malformed, e.g. "10; ex:Friend ...", and only one bad triple is parsed.

Could you include the tokenizer change and tests for ; and , shorthand?

Usually we add tests into kolibrie/tests/

Regarding the error with ML it was fixed, thanks for sharing :)

[ladroid]

The current implementation does not work unless tokenize_turtle_star_line also emits ; , and . as separate delimiter tokens outside IRIs/literals/quoted triples. With only the parse_turtle change, the object becomes malformed, e.g. "10; ex:Friend ...", and only one bad triple is parsed.