Add regex analyser action handler to execute APIv4 actions

[dontub]

This adds an regex analyser action handler for the action "api4" to execute APIv4 actions. It is inspired by #513.

To use values from the pattern matches, the bank transaction (btx), the bank account (ba), or the party bank account (party_ba) the Symfony Expression Language is used. So it is not only possible to access a single value, but to do some mathematics and other things. Strings starting with @= (this is the prefix Symfony uses in config files) are interpreted as expressions. I decided to give expressions also a try in the result_map in addition to what was discussed in #513. In expressions in the result map the Result object is available as variable result. I'm not sure if we need and want to have two approaches. (This is something to be discussed.) With the expression language the expression @=result.first()['some_field'][0] ?? NULL could be used instead of the first filter.

The Symfony Expression Language can be extended with custom functions so we're not limited to the ones available by default. (We could even consider giving third party extensions the possibility to provide custom functions.)

Adapted example from #513:

{
  "comment": "Look for previous contribution with matching bank name and amount",
  "action": "api4",
  "api4": {
    "entity": "Contribution",
    "action": "get",
    "params": {
      "limit": 1,
      "orderBy": {
        "receive_date": "DESC"
      },
      "where": [
        [
          "Donor_Information.Bank_Name",
          "=",
          "@=purpose"
        ],
        [
          "total_amount",
          "=",
          "@=btx.amount"
        ]
      ]
    },
    "result_map": {
      "previous_contribution_id": "id",
      "contact_id": "contact_id"
      "financial_type_id": "@=result.first()['financial_type_id']",
    }
  }
},

Actually @=result.first()['financial_type_id'] is the same as just financial_type_id here with results limited to one in the API call. It's just meant as an example. In \Civi\Banking\Matcher\RegexAnalyser\ActionHandlers\Api4RegexAnalyserActionHandlerTest you can find expressions with operations (addition, string concatenation).

Note: The . is the object access operator in expressions which prevents using dots in pattern matches. In case there are values with . in the parsed data of btx, array access can be used: btx['foo.bar'].

I'd like to hear your opinions @ufundo, @jensschuppe.

Another thing I'm currently unsure about: What is the expected behavior when the APIv4 call returns no result, but a result map is defined. The current implementation doesn't set any value at all, though another option would be to set NULL. In case of expressions they might be evaluated with the empty result object.

systopia-reference: 30273

[jensschuppe]

I think this should come with detailed documentation, especially regarding the symfony expression language part, and that should actually include some useful example configurations as well. CiviBanking is already hard to configure and this feature is generic and powerful enough to be explained in detail.

Actually, reviewing is hard for me because of that - maybe let's even briefly talk through it in person @dontub.

[ufundo]

@dontub this is looking great to me, thanks for taking it forward.

I really like the idea of using Symfony Expression Language:

• not yet another custom bespoke magic civi thing that only 2 people understand...
• seems very powerful, without having to have lots of additional keys in the rule syntax
• less to document if you can point at existing documentation for that part 😁

What is the expected behavior when the APIv4 call returns no result,

Great question. I'm not sure of the answer but the current implementation doesn't set any value at all seems like a good safe default. Then further rules can handle that case (based on things not having been filled).

In case of expressions they might be evaluated with the empty result object.

I think the risk here is that you end up with one very complex rule - I think it might be preferable to steer people towards "api4 rule which gets some data and fills based on it", then "fallback rule which handles the case when api4 didnt match anything". It may be you want to fetch more/different data in the case the first api4 doesn't return results; and you might have a daisy chain of rules like this.

[ufundo]

one very complex rule

I think there's an argument the result_map is already on the verge of "too much in one rule", and you'd be better off fetching the same data multiple times in separate rules, and setting a single property each time. result_map felt like quite a natural/handleable bit of complexity to me, but would be interested in other opinions on this.

[dontub]

Thinking about your last comment I could imagine something like this:

• The values of the result_map may only contain an APIv4 field name.
• Add a boolean option only_first_result. If TRUE $result->first()[<field name>] ?? NULL is set, otherwise $result->column(<field name>). (We might consider allowing to set index_by for this.)
  Edit: Using only the first result might be defined as default and to use all results an option like use_all_results might be used.
• Add a new action handler to apply expressions. The might be defined like this:

  {
     "action": "expression",
      "to": <target>,
      "expression": <expression>
  }

  It will evaluate <expression> with Sf Expression Language and set the result to <target>.

Something we might want to change for performance reasons: \CRM_Banking_BAO_BankTransaction::setDataParsed() is called every time a value is set and executes an SQL UPDATE. This UPDATE should be done only once everything is finished.