-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.NET: Bug: Anthropic connector doesn't work for VertexAI #10012
Comments
After trying a hacky workaround by intercepting the request and editing the json before sending it I noticed a new error. The 'model' property is not allowed, so that also needs to be removed from the body. https://docs.anthropic.com/en/api/claude-on-vertex-ai |
We did drop the PR for Anthropic connector after bringing There are currently our Amazon Connector that supports usage of Anthropic models in the Bedrock platform and other community libraries that can be using in Semantic Kernel, like:
How to use a
|
public static IChatCompletionService AsChatCompletionService(this IChatClient client, IServiceProvider? serviceProvider = null) |
Usage example:
var mySemanticKernelChatCompletion = myAnthropicClient.AsChatCompletionService();
Valid code using OllamaClient
that implements IChatClient
from M.E.AI.
var chatService = ollamaClient.AsChatCompletionService(); |
Describe the bug
The Anthropic VertexAI needs the anthropic-version to be passed in the body (as anthropic_version) instead of the header, as is done with the Anthropic API.
(See documentation here: https://docs.anthropic.com/en/api/claude-on-vertex-ai)
In Vertex, anthropic_version is passed in the request body (rather than as a header), and must be set to the value vertex-2023-10-16.
This is not done in the connector logic, it reuses the AnthropicRequest model which doesn't have the anthropic_version in the body.
When trying to use the AddAnthropicVertextAIChatCompletion out of the box it returns
Sadly this seems to be a hard blocker since I see no good way to add this into the body, but I would love if anyone had a workaround.
UPDATE: After trying a hacky workaround by intercepting the request and editing the json before sending it I noticed a new error. The 'model' property is not allowed, so that also needs to be removed from the body.
Additional context
Not sure if it's a bug or intented, but unlike the GeminiVertexAi connector, the AddAnthropicVertextAIChatCompletion (it also has a typo) isn't creating the full endpoint for you. So you need to create the full URL like so:
Which I think could be improved to be more intuitive, and be similar to the Gemini VertexAI connector.
The text was updated successfully, but these errors were encountered: