[FLINK-38882][table][python] Introduce rules for async python scalar function #27395
Conversation
| bool profile_enabled = 4; | ||
| repeated JobParameter job_parameters = 5; | ||
| // Async execution configuration for async scalar functions | ||
| int32 async_buffer_capacity = 6; // Max number of concurrent async operations |
There was a problem hiding this comment.
I am curious how we have come up with these numbers. async_max_attempts = 9 seems a strange amount?
There was a problem hiding this comment.
It just represents the index of this variable in the message.
| bool async_retry_enabled = 8; // Whether retry is enabled | ||
| int32 async_max_attempts = 9; // Maximum number of retry attempts | ||
| int64 async_retry_delay_ms = 10; // Delay between retries in milliseconds | ||
| } |
There was a problem hiding this comment.
this is a bool - I assume the value should not be 8.
There was a problem hiding this comment.
It just represents the index of this variable in the message.
| int64 async_timeout_ms = 7; // Timeout in milliseconds for async operations | ||
| // Async retry strategy configuration | ||
| bool async_retry_enabled = 8; // Whether retry is enabled | ||
| int32 async_max_attempts = 9; // Maximum number of retry attempts |
There was a problem hiding this comment.
I suggest including retry in the name to be consistent with the other variables. async_max_retry_attempts
| repeated OverWindow windows = 3; | ||
| bool profile_enabled = 4; | ||
| repeated JobParameter job_parameters = 5; | ||
| // Async execution configuration for async scalar functions |
There was a problem hiding this comment.
I suspect we need to include these values in the docs, with a write up of this capability.
There was a problem hiding this comment.
6 is not a capacity value, it represents the index of this variable in the message UserDefinedFunctions (a specification requirement of protobuf)
| PythonCalcSplitRule.REWRITE_PROJECT, | ||
| PythonMapRenameRule.INSTANCE, | ||
| PythonMapMergeRule.INSTANCE, | ||
| // Splits Python calc which contains Python async scalar functions and other Python functions |
There was a problem hiding this comment.
For my understanding : why do we need Python specific rules? What in the existing rules does not work for Python?
There was a problem hiding this comment.
Good point. Removed.
What is the purpose of the change
This pull request introduces the necessary rules and nodes to support async python scalar function
Brief change log
(for example:)
Verifying this change
This change added tests and can be verified as follows:
(example:)
Does this pull request potentially affect one of the following parts:
@Public(Evolving): (yes / no)Documentation