Issue with OllamaSharp endpoint?

[ch4uTR]

Hi!

I've been learning C# and I tried the new ASP.NET AI Chat Web Application template.

When I send a request to http://localhost:11434/api/generate using Postman, I get a valid response because the llama3.2:latest model is running.

However, when I use this template, which relies on OllamaSharp, it throws an exception:

System.Net.Http.HttpRequestException: 'Response status code does not indicate success: 404 (Not Found).'

To investigate, I wrote a small code snippet in Program.cs. When I run it, the request succeeds:

          var client = new HttpClient();
          client.BaseAddress = new Uri("http://localhost:11434");
          
          var payload = new
          {
              model = "llama3.2:latest",
              prompt = "Hello"
          };
          
          var content = new StringContent(JsonSerializer.Serialize(payload), Encoding.UTF8, "application/json");
          var response = await client.PostAsync("api/generate", content);
          var result = await response.Content.ReadAsStringAsync();
          Console.WriteLine(result);

I’m still learning C#, so I might be missing something on my side, but I wonder if there could be an issue with the endpoints that OllamaSharp uses internally when sending requests.

PS: I’m a student and this is for learning purposes. I would really appreciate any guidance or advice you could offer.

[awaescher]

Hi, I agree this is strange. So would you mind sharing some code to investigate?
While we have no code to speak about, let me follow up on this ...

That means this code won't work in your side?

var uri = new Uri("http://localhost:11434");
var ollama = new OllamaApiClient(uri);

var models = await ollama.ListLocalModelsAsync();

This should be the simplest call to the Ollama API, without streaming or anything. If you get the local models back, you set up everything correctly. If not, we need to investigate that.

I could also highly recommend cloning this repository and running dotnet run --project demo which starts my demo console project. It has a lot of samples how to use this library. If you use Ollama on the same machine on the default port 11434, you won't need to enter anything, just hit [Enter] when the console asks for a machine name or endpoint url.

[ch4uTR]

Hi,

Thank you for your reply. Ofcourse, here is the program.cs :

  `using LocalChatApp.Components;
  using LocalChatApp.Services;
  using LocalChatApp.Services.Ingestion;
  using Microsoft.Extensions.AI;
  using OllamaSharp;
  using System.Text;
  using System.Text.Json;
  
  var builder = WebApplication.CreateBuilder(args);
  builder.Services.AddRazorComponents().AddInteractiveServerComponents();
  
  IChatClient chatClient = new OllamaApiClient(new Uri("http://localhost:11434"),
      "llama3.2:latest");
  IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator = new OllamaApiClient(new Uri("http://localhost:11434/"),
      "all-minilm");
  
  var vectorStorePath = Path.Combine(AppContext.BaseDirectory, "vector-store.db");
  var vectorStoreConnectionString = $"Data Source={vectorStorePath}";
  builder.Services.AddSqliteCollection<string, IngestedChunk>("data-localchatapp-chunks", vectorStoreConnectionString);
  builder.Services.AddSqliteCollection<string, IngestedDocument>("data-localchatapp-documents", vectorStoreConnectionString);
  
  builder.Services.AddScoped<DataIngestor>();
  builder.Services.AddSingleton<SemanticSearch>();
  builder.Services.AddChatClient(chatClient).UseFunctionInvocation().UseLogging();
  builder.Services.AddEmbeddingGenerator(embeddingGenerator);
  
  var app = builder.Build();
  
  // Configure the HTTP request pipeline.
  if (!app.Environment.IsDevelopment())
  {
      app.UseExceptionHandler("/Error", createScopeForErrors: true);
      // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
      app.UseHsts();
  }
  
  app.UseHttpsRedirection();
  app.UseAntiforgery();
  
  app.UseStaticFiles();
  app.MapRazorComponents<App>()
      .AddInteractiveServerRenderMode();
  
  
  
  
  // By default, we ingest PDF files from the /wwwroot/Data directory. You can ingest from
  // other sources by implementing IIngestionSource.
  // Important: ensure that any content you ingest is trusted, as it may be reflected back
  // to users or could be a source of prompt injection risk.
  await DataIngestor.IngestDataAsync(
      app.Services,
      new PDFDirectorySource(Path.Combine(builder.Environment.WebRootPath, "Data")));
  
  app.Run();
  `

And I pasted your code snippet into program.cs and the result was this. I suppose it couldn't reach those models.

    `OllamaSharp.Models.Model[]
    info: LocalChatApp.Services.Ingestion.DataIngestor[0]
          Ingestion is up-to-date
    info: Microsoft.Hosting.Lifetime[14]
          Now listening on: https://localhost:7299
    info: Microsoft.Hosting.Lifetime[14]
          Now listening on: http://localhost:5163
    info: Microsoft.Hosting.Lifetime[0]
          Application started. Press Ctrl+C to shut down.
    info: Microsoft.Hosting.Lifetime[0]
          Hosting environment: Development
    info: Microsoft.Hosting.Lifetime[0]
          Content root path: C:\Users\enesy\source\repos\ecommerce\LocalChatApp
    `

In addition, I'll clone the repository as you recommended.

[awaescher]

I don't know how you logged the models from ListLocalModels() but this looks like a valid array in the logs