Update LLM API Router Service
@@ -32,6 +32,10 @@ configuration is provided through a `LLMApiRouterConfiguration` object.
|
||||
`AllLLMServicesUnavailableException`.
|
||||
* The request is routed to the selected `LLMService`.
|
||||
* This call is wrapped in the `autoRetry` method
|
||||
* depending on the endpoint, the `shortTimeout` parameter is set
|
||||
* `true` for `processQuery`
|
||||
* `false` for `generateSummary`
|
||||
|
||||
|
||||
> **Note** The service selection mechanism is implemented in the `currentService` property.
|
||||
> This property is evaluated for each access, so it should always return an `LLMService` that is flagged as `operational`
|
||||
|
||||
Reference in New Issue
Block a user