Check if ASR grammars exist before access by danslavetskiy · Pull Request #24 · freeswitch/mod_unimrcp

danslavetskiy · 2025-02-21T18:30:03Z

Hi Team,

unfortunately #20 didn't fix the issue and we still have crashes on production. The main reason is that there are situations when grammars get destroyed and then other functions assume that they exist and locking here does not solve the problem. So i've quickly added some checks to see if this really helps and it does..

i'm new to the project and don't know how to do it better:) maybe the issue is layer upper - in the code that calls mod_unimrcp.

… speech_channel_destroy

danslavetskiy · 2025-02-21T18:33:56Z

mod_unimrcp.c

 {
 	switch_mutex_lock(schannel->mutex);
-	if (!zstr(param) && val != NULL) {
+	if (!zstr(param) && val != NULL && schannel->params) {


get's destroyed in speech_channel_destroy

mod_unimrcp/mod_unimrcp.c

Line 949 in 5eaae25

switch_core_hash_destroy(&schannel->params);

then used in

mod_unimrcp/mod_unimrcp.c

Line 1471 in 5eaae25

switch_core_hash_insert(schannel->params, p, v);

below core dump

#0 0x00007f7539767602 in hash (h=0x0, k=0x7f722c0a91a0) at ./src/include/private/switch_hashtable_private.h:53 #1 switch_hashtable_insert_destructor (h=h@entry=0x0, k=k@entry=0x7f722c0a91a0, v=v@entry=0x7f72001ea0d0, flags=flags@entry=(HASHTABLE_FLAG_FREE_KEY | HASHTABLE_DUP_CHECK), destructor=destructor@entry=0x0) at src/switch_hashtable.c:197 #2 0x00007f7539663944 in switch_core_hash_insert_destructor (hash=0x0, key=key@entry=0x7f72001ea0b8 "speech-language", data=data@entry=0x7f72001ea0d0, destructor=destructor@entry=0x0) at src/switch_core_hash.c:148 #3 0x00007f753520cc70 in speech_channel_set_param (schannel=0x7f722c17d608, param=<optimized out>, val=0x7f722c08cba2 "pl-PL") at mod_unimrcp.c:1471 #4 0x00007f7539659965 in switch_core_asr_load_grammar (ah=0x7f722c17d560, grammar=grammar@entry=0x7f72001e9f2a "{ speech-language=pl-PL, session-id=60ea7a18-3fd3-4ba3-bd57-695a77712c8f, ai.lekta.asr.id=dictation, ai.lekta.asr.trace-id=60ea7a18-3fd3-4ba3-bd57-695a77712c8f, ai.lekta.asr.external-id=42ed1d9d-6445-"..., name=name@entry=0x7f75399a4de7 "") at src/switch_core_asr.c:151 ....

danslavetskiy · 2025-02-21T18:36:47Z

mod_unimrcp.c

-	if ((status = grammar_create(&g, name, type, data, schannel->memory_pool)) == SWITCH_STATUS_SUCCESS) {
-		recognizer_data_t *r = (recognizer_data_t *) schannel->data;
+	recognizer_data_t *r = (recognizer_data_t *) schannel->data;
+	if (r->grammars && (status = grammar_create(&g, name, type, data, schannel->memory_pool)) == SWITCH_STATUS_SUCCESS) {


grammars get destroyed in recog_asr_close

mod_unimrcp/mod_unimrcp.c

Line 3429 in 5eaae25

switch_core_hash_destroy(&r->grammars);

then used in

mod_unimrcp/mod_unimrcp.c

Line 2433 in 5eaae25

switch_core_hash_insert(r->grammars, g->name, g);

Thread 1 (Thread 0x7fdd40baf6c0 (LWP 34013)): #0 0x00007fdd985c1602 in hash (h=0x0, k=0x7fdb50191690) at ./src/include/private/switch_hashtable_private.h:53 #1 switch_hashtable_insert_destructor (h=h@entry=0x0, k=k@entry=0x7fdb50191690, v=v@entry=0x7fda900866a0, flags=flags@entry=(HASHTABLE_FLAG_FREE_KEY | HASHTABLE_DUP_CHECK), destructor=destructor@entry=0x0) at src/switch_hashtable.c:197 #2 0x00007fdd984bd944 in switch_core_hash_insert_destructor (hash=0x0, key=<optimized out>, data=data@entry=0x7fda900866a0, destructor=destructor@entry=0x0) at src/switch_core_hash.c:148 #3 0x00007fdd66a320c7 in recog_channel_load_grammar (data=0x7fdb5004a870 "session:de3cd22c-cbb2-4e89-99ad-df98865999a2", type=GRAMMAR_TYPE_URI, name=0x7fda90086678 "de3cd22c-cbb2-4e89-99ad-df98865999a2", schannel=0x7fdb500b0328) at mod_unimrcp.c:2433 #4 recog_asr_load_grammar (ah=<optimized out>, grammar=<optimized out>, name=0x7fda90086678 "de3cd22c-cbb2-4e89-99ad-df98865999a2") at mod_unimrcp.c:3313 ...

so we check if exists in below cases

same for enabled_grammars

danslavetskiy · 2025-02-21T18:44:05Z

mod_unimrcp.c

 	switch_log_printf(SWITCH_CHANNEL_UUID_LOG(schannel->session_uuid), SWITCH_LOG_DEBUG, "(%s) Disabling all grammars\n", schannel->name);

 	switch_mutex_lock(schannel->mutex);
-	switch_core_hash_destroy(&r->enabled_grammars);


Thread 1 (Thread 0x7f923fe546c0 (LWP 34592)): #0 __GI_abort () at ./stdlib/abort.c:107 #1 0x00007f931fa92395 in __assert_fail_base (fmt=0x7f931fc06a90 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7f932001b7a8 "hash != ((void *)0) && *hash != ((void *)0)", file=file@entry=0x7f932001b76a "src/switch_core_hash.c", line=line@entry=51, function=function@entry=0x7f932001b8f0 <__PRETTY_FUNCTION__.6> "switch_core_hash_destroy") at ./assert/assert.c:92 #2 0x00007f931faa0eb2 in __GI___assert_fail (assertion=assertion@entry=0x7f932001b7a8 "hash != ((void *)0) && *hash != ((void *)0)", file=file@entry=0x7f932001b76a "src/switch_core_hash.c", line=line@entry=51, function=function@entry=0x7f932001b8f0 <__PRETTY_FUNCTION__.6> "switch_core_hash_destroy") at ./assert/assert.c:101 #3 0x00007f931fcd661a in switch_core_hash_destroy (hash=hash@entry=0x7f90e4061820) at src/switch_core_hash.c:51 #4 0x00007f92ef409135 in recog_channel_disable_all_grammars (schannel=schannel@entry=0x7f90e4140498) at mod_unimrcp.c:2543 #5 0x00007f92ef410c77 in recog_asr_load_grammar (ah=<optimized out>, grammar=<optimized out>, name=0x7f8fb4149e18 "f88801fb-f8f2-456f-b147-35319489e7c3") at mod_unimrcp.c:3321

danslavetskiy · 2025-02-21T18:47:42Z

mod_unimrcp.c

+	if (r->enabled_grammars) {
+		switch_core_hash_destroy(&r->enabled_grammars);
+		switch_core_hash_init(&r->enabled_grammars);
+		status = SWITCH_STATUS_SUCCESS;


so that we omit recog_channel_enable_grammar and recog_channel_start in

mod_unimrcp/mod_unimrcp.c

Line 3324 in 5eaae25

if (recog_channel_disable_all_grammars(schannel) != SWITCH_STATUS_SUCCESS) {

danslavetskiy · 2025-02-26T10:03:55Z

Hi @andywolk @morbit85 what do you think about these changes? The issue occurs when there are CPU spikes on the machine with ASR (which is behind MRCP).

danielslavetskiy and others added 6 commits February 19, 2025 16:18

add conditions for concurrent grammar handling

a495ce7

add conditions for concurrent grammar handling

937dd9a

check if schannel->params exist, as thay could have been destroyed in…

a231761

… speech_channel_destroy

improve

7c5cef3

improve

789a6f6

Merge branch 'freeswitch:main' into asr-grammar-conditions

39c3b5e

danslavetskiy commented Feb 21, 2025

View reviewed changes

danslavetskiy changed the title ~~ASR grammar conditions~~ Check if ASR grammars exist before access Feb 21, 2025

andywolk requested review from andywolk and morbit85 February 26, 2025 10:32

Merge branch 'main' into asr-grammar-conditions

6aaca77

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check if ASR grammars exist before access#24

Check if ASR grammars exist before access#24
danslavetskiy wants to merge 7 commits intofreeswitch:mainfrom
danslavetskiy:asr-grammar-conditions

danslavetskiy commented Feb 21, 2025 •

edited

Loading

Uh oh!

danslavetskiy Feb 21, 2025 •

edited

Loading

Uh oh!

danslavetskiy Feb 21, 2025 •

edited

Loading

Uh oh!

danslavetskiy Feb 21, 2025

Uh oh!

danslavetskiy Feb 21, 2025

Uh oh!

danslavetskiy commented Feb 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

danslavetskiy commented Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danslavetskiy Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danslavetskiy Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danslavetskiy Feb 21, 2025

Choose a reason for hiding this comment

Uh oh!

danslavetskiy Feb 21, 2025

Choose a reason for hiding this comment

Uh oh!

danslavetskiy commented Feb 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

danslavetskiy commented Feb 21, 2025 •

edited

Loading

danslavetskiy Feb 21, 2025 •

edited

Loading

danslavetskiy Feb 21, 2025 •

edited

Loading