Failed to start

[malisancube]

Not sure if this is related to anything, but failed to start.

Version 17.14.0 Preview 1.0
System Information
------------------
      Time of this report: 2/16/2025, 00:04:01
             Machine name: 000000
               Machine Id: {D4819370-5028-482C-83B0-B0FA7424CDD8}
         Operating System: Windows 11 Pro 64-bit (10.0, Build 26100) (26100.ge_release.240331-1435)
                 Language: English (Regional Setting: English)
      System Manufacturer: HP
             System Model: OMEN 30L Desktop GT13-0xxx
                     BIOS: F.24 (type: UEFI)
                Processor: Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz (16 CPUs), ~3.7GHz
                   Memory: 57344MB RAM
      Available OS Memory: 57206MB RAM
                Page File: 27733MB used, 33057MB available
              Windows Dir: C:\WINDOWS
          DirectX Version: DirectX 12
      DX Setup Parameters: Not found
         User DPI Setting: 120 DPI (125 percent)
       System DPI Setting: 288 DPI (300 percent)
          DWM DPI Scaling: Disabled
                 Miracast: Available, no HDCP
Microsoft Graphics Hybrid: Not Supported
 DirectX Database Version: 1.6.6
   Auto Super Res Version: Unknown
       System Mux Support: Mux Support Inactive - Ok
           Mux Target GPU: dGPU
    Mux Incompatible List: 
           DxDiag Version: 10.00.26100.3037 64bit Unicode

DocDBTrace Information: 0 : GlobalEndpointManager: StartLocationBackgroundRefreshWithTimer() - Invoking refresh
DocDBTrace Information: 0 : Current WriteEndpoints = (http://localhost:63478/) ReadEndpoints = (http://localhost:63478/)
DocDBTrace Information: 0 : GlobalEndpointManager: StartLocationBackgroundRefreshWithTimer() refreshing locations
Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler: Error: could not connect a container to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 355, "Container": "09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0", "Network": "default-aspire-network-06ut3s4t58", "error": "container 09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0 is not connected to network daf0bdb27b7561a842d9efc922ce8e682d5505b93b7ef4459debb097d59201be"}

The thread '.NET TP Worker' (27860) has exited with code 0 (0x0).
The thread '.NET TP Worker' (32156) has exited with code 0 (0x0).
The thread '.NET TP Worker' (35744) has exited with code 0 (0x0).
The thread '.NET TP Worker' (32896) has exited with code 0 (0x0).
The thread '.NET TP Worker' (27528) has exited with code 0 (0x0).
Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler: Error: could not connect a container to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 356, "Container": "09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0", "Network": "default-aspire-network-06ut3s4t58", "error": "container 09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0 is not connected to network daf0bdb27b7561a842d9efc922ce8e682d5505b93b7ef4459debb097d59201be"}

The thread '.NET TP Worker' (33184) has exited with code 0 (0x0).
Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler: Error: could not connect a container to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 357, "Container": "09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0", "Network": "default-aspire-network-06ut3s4t58", "error": "container 09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0 is not connected to network daf0bdb27b7561a842d9efc922ce8e682d5505b93b7ef4459debb097d59201be"}

The thread '.NET TP Worker' (9644) has exited with code 0 (0x0).
Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler: Error: could not connect a container to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 358, "Container": "09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0", "Network": "default-aspire-network-06ut3s4t58", "error": "container 09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0 is not connected to network daf0bdb27b7561a842d9efc922ce8e682d5505b93b7ef4459debb097d59201be"}

Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler: Error: could not connect a container to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 359, "Container": "09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0", "Network": "default-aspire-network-06ut3s4t58", "error": "container 09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0 is not connected to network daf0bdb27b7561a842d9efc922ce8e682d5505b93b7ef4459debb097d59201be"}

Aspire.Hosting.Dcp.dcpctrl.NetworkReconciler: Error: could not connect a container to the network	{"NetworkName": {"name":"default-aspire-network"}, "Reconciliation": 360, "Container": "09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0", "Network": "default-aspire-network-06ut3s4t58", "error": "container 09507fa058a784a895e1a3c4e75943a9e4175b57f4379113fdaa4b364dad2dd0 is not connected to network daf0bdb27b7561a842d9efc922ce8e682d5505b93b7ef4459debb097d59201be"}

[davidfowl]

Is your version of docker updated?

@karolz-ms

[davidfowl]

Wait, could this be a GPU issue?

[davidfowl]

If you comment out

aspire-ai-chat-demo/AIChat.AppHost/ModelExtensions.cs

Line 22 in 9d622cc

.WithGPUSupport()

, does it work?

[274188A]

ah ha! - yes it works now. I previously opened then closed an issue on this.

[malisancube]

Yes. GPU issue.

There was another issue as follows.

From VS

The thread '.NET TP Worker' (7072) has exited with code 0 (0x0).
DocDBTrace Information: 0 : GlobalEndpointManager: Fail to reach gateway endpoint http://localhost:53304/, Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: ServiceUnavailable (503); Substatus: 20003; ActivityId: 7b87fc88-0450-44fc-b2bf-6804b971c58e; Reason: (GatewayStoreClient Request Timeout. Start Time UTC:2/16/2025 8:00:19 PM; Total Duration:31035.2585 Ms; Request Timeout 20000 Ms; Http Client Timeout:65000 Ms; Activity id: 7b87fc88-0450-44fc-b2bf-6804b971c58e;);
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken, DocumentServiceRequest documentServiceRequest)
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken, DocumentServiceRequest documentServiceRequest)
   at Microsoft.Azure.Cosmos.GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint)
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAndUpdateAccountPropertiesAsync(Uri endpoint)
--- Cosmos Diagnostics ---{"Summary":{},"name":"Account Read","start datetime":"2025-02-16T20:00:19.468Z","duration in milliseconds":31040.9413}
DocDBTrace Error: 0 : Operation will NOT be retried. Current attempt 0, Exception: Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: ServiceUnavailable (503); Substatus: 20003; ActivityId: 7b87fc88-0450-44fc-b2bf-6804b971c58e; Reason: (GatewayStoreClient Request Timeout. Start Time UTC:2/16/2025 8:00:19 PM; Total Duration:31035.2585 Ms; Request Timeout 20000 Ms; Http Client Timeout:65000 Ms; Activity id: 7b87fc88-0450-44fc-b2bf-6804b971c58e;);
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken, DocumentServiceRequest documentServiceRequest)
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetOnlyGlobalEndpointAsync()
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAccountPropertiesAsync()
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetDatabaseAccountFromAnyLocationsAsync(Uri defaultEndpoint, IList`1 locations, IList`1 accountInitializationCustomEndpoints, Func`2 getDatabaseAccountFn, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()
   at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync[TParam,TPolicy](Func`1 callbackMethod, Func`3 callbackMethodWithParam, Func`2 callbackMethodWithPolicy, TParam param, IRetryPolicy retryPolicy, IRetryPolicy`1 retryPolicyWithArg, Func`1 inBackoffAlternateCallbackMethod, Func`2 inBackoffAlternateCallbackMethodWithPolicy, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)
--- Cosmos Diagnostics ---{"Summary":{},"name":"Account Read","start datetime":"2025-02-16T20:00:19.468Z","duration in milliseconds":31040.9413} 
DocDBTrace Error: 0 : AsyncCacheNonBlocking Failed GetAsync with key: InitTaskKey, Exception: Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: ServiceUnavailable (503); Substatus: 20003; ActivityId: 7b87fc88-0450-44fc-b2bf-6804b971c58e; Reason: (GatewayStoreClient Request Timeout. Start Time UTC:2/16/2025 8:00:19 PM; Total Duration:31035.2585 Ms; Request Timeout 20000 Ms; Http Client Timeout:65000 Ms; Activity id: 7b87fc88-0450-44fc-b2bf-6804b971c58e;);
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken, DocumentServiceRequest documentServiceRequest)
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetOnlyGlobalEndpointAsync()
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAccountPropertiesAsync()
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetDatabaseAccountFromAnyLocationsAsync(Uri defaultEndpoint, IList`1 locations, IList`1 accountInitializationCustomEndpoints, Func`2 getDatabaseAccountFn, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()
   at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync[TParam,TPolicy](Func`1 callbackMethod, Func`3 callbackMethodWithParam, Func`2 callbackMethodWithPolicy, TParam param, IRetryPolicy retryPolicy, IRetryPolicy`1 retryPolicyWithArg, Func`1 inBackoffAlternateCallbackMethod, Func`2 inBackoffAlternateCallbackMethodWithPolicy, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.ShouldRetryResult.ThrowIfDoneTrying(ExceptionDispatchInfo capturedException)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync[TParam,TPolicy](Func`1 callbackMethod, Func`3 callbackMethodWithParam, Func`2 callbackMethodWithPolicy, TParam param, IRetryPolicy retryPolicy, IRetryPolicy`1 retryPolicyWithArg, Func`1 inBackoffAlternateCallbackMethod, Func`2 inBackoffAlternateCallbackMethodWithPolicy, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync[TParam,TPolicy](Func`1 callbackMethod, Func`3 callbackMethodWithParam, Func`2 callbackMethodWithPolicy, TParam param, IRetryPolicy retryPolicy, IRetryPolicy`1 retryPolicyWithArg, Func`1 inBackoffAlternateCallbackMethod, Func`2 inBackoffAlternateCallbackMethodWithPolicy, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Cosmos.AsyncCacheNonBlocking`2.GetAsync(TKey key, Func`2 singleValueInitFunc, Func`2 forceRefresh)
--- Cosmos Diagnostics ---{"Summary":{},"name":"Account Read","start datetime":"2025-02-16T20:00:19.468Z","duration in milliseconds":31040.9413}
DocDBTrace Warning: 0 : EnsureValidClientAsync initializeTask failed Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: ServiceUnavailable (503); Substatus: 20003; ActivityId: 7b87fc88-0450-44fc-b2bf-6804b971c58e; Reason: (GatewayStoreClient Request Timeout. Start Time UTC:2/16/2025 8:00:19 PM; Total Duration:31035.2585 Ms; Request Timeout 20000 Ms; Http Client Timeout:65000 Ms; Activity id: 7b87fc88-0450-44fc-b2bf-6804b971c58e;);
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken, DocumentServiceRequest documentServiceRequest)
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetOnlyGlobalEndpointAsync()
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAccountPropertiesAsync()
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetDatabaseAccountFromAnyLocationsAsync(Uri defaultEndpoint, IList`1 locations, IList`1 accountInitializationCustomEndpoints, Func`2 getDatabaseAccountFn, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()
   at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync[TParam,TPolicy](Func`1 callbackMethod, Func`3 callbackMethodWithParam, Func`2 callbackMethodWithPolicy, TParam param, IRetryPolicy retryPolicy, IRetryPolicy`1 retryPolicyWithArg, Func`1 inBackoffAlternateCallbackMethod, Func`2 inBackoffAlternateCallbackMethodWithPolicy, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.ShouldRetryResult.ThrowIfDoneTrying(ExceptionDispatchInfo capturedException)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync[TParam,TPolicy](Func`1 callbackMethod, Func`3 callbackMethodWithParam, Func`2 callbackMethodWithPolicy, TParam param, IRetryPolicy retryPolicy, IRetryPolicy`1 retryPolicyWithArg, Func`1 inBackoffAlternateCallbackMethod, Func`2 inBackoffAlternateCallbackMethodWithPolicy, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync[TParam,TPolicy](Func`1 callbackMethod, Func`3 callbackMethodWithParam, Func`2 callbackMethodWithPolicy, TParam param, IRetryPolicy retryPolicy, IRetryPolicy`1 retryPolicyWithArg, Func`1 inBackoffAlternateCallbackMethod, Func`2 inBackoffAlternateCallbackMethodWithPolicy, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Cosmos.AsyncCacheNonBlocking`2.GetAsync(TKey key, Func`2 singleValueInitFunc, Func`2 forceRefresh)
   at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync(ITrace trace)
--- Cosmos Diagnostics ---{"Summary":{},"name":"Account Read","start datetime":"2025-02-16T20:00:19.468Z","duration in milliseconds":31040.9413}
The thread '.NET TP Worker' (20824) has exited with code 0 (0x0).
The thread '.NET TP Worker' (30556) has exited with code 0 (0x0).
The thread '.NET TP Worker' (12620) has exited with code 0 (0x0).
The thread 24480 has exited with code 0 (0x0).
The thread '.NET TP Worker' (29536) has exited with code 0 (0x0).
The thread '.NET TP Worker' (3972) has exited with code 0 (0x0).
The thread '.NET TP Worker' (27880) has exited with code 0 (0x0).
The thread '.NET TP Worker' (32844) has exited with code 0 (0x0).
The thread '.NET TP Worker' (12916) has exited with code 0 (0x0).
Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy: Error: Error handling TCP connection	{"Service": {"name":"cosmos-emulator"}, "error": "no endpoints configured"}

Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler.Proxy: Error: Error handling TCP connection	{"Service": {"name":"cosmos-emulator"}, "error": "no endpoints configured"}

From Dashboard

[ed](https://localhost:17255/structuredlogs)
[Traces](https://localhost:17255/traces)
[Metrics](https://localhost:17255/metrics)
Console logs
Resource



Watching logs...


2025-02-16T22:59:11 1efb9851240fb2a47adec5f4eeadc02bdcdd9976eb8c3fd19c992af80e03739c
2025-02-16T22:59:16
 cat: /version.txt: No such file or directory
2025-02-16T22:59:16 Release Version: 
2025-02-16T22:59:16
 cp: cannot stat '/data/cosmosdev.crt': No such file or directory
2025-02-16T22:59:16
 cp: cannot stat '/data/cosmosdevroot.crt': No such file or directory
2025-02-16T22:59:16
 cp: cannot stat '/data/cosmosdev.key': No such file or directory
2025-02-16T23:00:18 Stopping any existing postgres servers�(B
2025-02-16T23:00:18 pg_ctl: PID file "/data/db/postmaster.pid" does not exist
2025-02-16T23:00:18 Is server running?
2025-02-16T23:00:18 Stopped all PG instances on /data/db
2025-02-16T23:00:18 pg_ctl: directory "/home/cosmosdev/workernode*" does not exist
2025-02-16T23:00:18 Stopped all PG instances on //home/cosmosdev/workernode*/
2025-02-16T23:00:18 Starting postgres in /data/db
2025-02-16T23:00:18 Calling: pg_ctl start -D /data/db -o "-p 9712 -i " -l /logs/pglog.log
2025-02-16T23:00:18 waiting for server to start.... stopped waiting
2025-02-16T23:00:18 pg_ctl: could not start server
2025-02-16T23:00:18 Examine the log output.
2025-02-16T23:00:18 Postgres failed to start

[davidfowl]

What does the dashboard look like?

[malisancube]

As follows

[davidfowl]

Cosmos never starts?

[malisancube]

Cosmos never starts?

Yes. Does not start. If I start it manually I get the following logs.

2025-02-17T23:30:29 9184d93bf24f47b66a0b70651f2882b31305873d0dec393b74aa406ccdd31710
2025-02-17T23:30:33
 cat: /version.txt: No such file or directory
2025-02-17T23:30:33 Release Version: 
2025-02-17T23:30:33
 cp: cannot stat '/data/cosmosdev.crt': No such file or directory
2025-02-17T23:30:33
 cp: cannot stat '/data/cosmosdevroot.crt': No such file or directory
2025-02-17T23:30:33
 cp: cannot stat '/data/cosmosdev.key': No such file or directory
2025-02-17T23:31:35 Stopping any existing postgres servers�(B
2025-02-17T23:31:35 pg_ctl: PID file "/data/db/postmaster.pid" does not exist
2025-02-17T23:31:35 Is server running?
2025-02-17T23:31:35 Stopped all PG instances on /data/db
2025-02-17T23:31:35 pg_ctl: directory "/home/cosmosdev/workernode*" does not exist
2025-02-17T23:31:35 Stopped all PG instances on //home/cosmosdev/workernode*/
2025-02-17T23:31:35 Starting postgres in /data/db
2025-02-17T23:31:35 Calling: pg_ctl start -D /data/db -o "-p 9712 -i " -l /logs/pglog.log
2025-02-17T23:31:35 waiting for server to start.... stopped waiting
2025-02-17T23:31:35 pg_ctl: could not start server
2025-02-17T23:31:35 Examine the log output.
2025-02-17T23:31:35 Postgres failed to start

[davidfowl]

Some things you can try:

1. Try deleting the container image and letting it re-pull a new image on run.
2. Try deleting the volume and restarting.

[davidfowl]

I ran into the same thing and these steps helped

[WretchedDade]

Along the lines of the original issue, I was getting an error when ollama was trying to start, it was failing continuously with "auto-detected mode as 'legacy'\nnvidia-container-cli: initialization error: wsl environment detected but no adapters were found: unknown\nobject not found\ncontainer not found". Commenting out EnableGPUSupport() fixed this issue.