{"@t":"2025-03-14T02:01:04.2204897Z","@mt":"{State:l}","@r":["Server be77b3cf-aa78-4693-a430-1d5f4756497b encountered an exception while sending heartbeat"],"@l":"Warning","@x":"Microsoft.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server: Could not open a connection to SQL Server)\n at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)\n at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)\n at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection()\n at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func)\n at Hangfire.Server.ServerHeartbeatProcess.Execute(BackgroundProcessContext context)\nClientConnectionId:00000000-0000-0000-0000-000000000000","State":"Server be77b3cf-aa78-4693-a430-1d5f4756497b encountered an exception while sending heartbeat","SourceContext":"Hangfire.Server.ServerHeartbeatProcess","ThreadId":14,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:01:04.2201845Z","@mt":"{State:l}","@r":["Execution DelayedJobScheduler is in the Failed state now due to an exception, execution will be retried no more than in 00:00:09"],"@l":"Error","@x":"Microsoft.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server: Could not open a connection to SQL Server)\n at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)\n at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)\n at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection()\n at Hangfire.SqlServer.SqlServerConnection.AcquireLock(String resource, TimeSpan timeout)\n at Hangfire.Server.DelayedJobScheduler.UseConnectionDistributedLock[T](JobStorage storage, Func`2 action)\n at Hangfire.Server.DelayedJobScheduler.Execute(BackgroundProcessContext context)\n at Hangfire.Server.BackgroundProcessDispatcherBuilder.ExecuteProcess(Guid executionId, Object state)\n at Hangfire.Processing.BackgroundExecution.Run(Action`2 callback, Object state)\nClientConnectionId:00000000-0000-0000-0000-000000000000","State":"Execution DelayedJobScheduler is in the Failed state now due to an exception, execution will be retried no more than in 00:00:09","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":40,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:01:04.2201843Z","@mt":"{State:l}","@r":["Execution Worker is in the Failed state now due to an exception, execution will be retried no more than in 00:05:00"],"@l":"Error","@x":"Microsoft.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server: Could not open a connection to SQL Server)\n at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)\n at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)\n at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection()\n at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func)\n at Hangfire.SqlServer.SqlServerJobQueue.DequeueUsingSlidingInvisibilityTimeout(String[] queues, CancellationToken cancellationToken)\n at Hangfire.SqlServer.SqlServerJobQueue.Dequeue(String[] queues, CancellationToken cancellationToken)\n at Hangfire.SqlServer.SqlServerConnection.FetchNextJob(String[] queues, CancellationToken cancellationToken)\n at Hangfire.Server.Worker.Execute(BackgroundProcessContext context)\n at Hangfire.Server.BackgroundProcessDispatcherBuilder.ExecuteProcess(Guid executionId, Object state)\n at Hangfire.Processing.BackgroundExecution.Run(Action`2 callback, Object state)\nClientConnectionId:00000000-0000-0000-0000-000000000000","State":"Execution Worker is in the Failed state now due to an exception, execution will be retried no more than in 00:05:00","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":32,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:01:04.2201843Z","@mt":"{State:l}","@r":["Execution RecurringJobScheduler is in the Failed state now due to an exception, execution will be retried no more than in 00:00:09"],"@l":"Error","@x":"Microsoft.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server: Could not open a connection to SQL Server)\n at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)\n at Microsoft.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, SqlConnectionString connectionOptions, Boolean withFailover)\n at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)\n at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance)\n at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken, DbConnectionPool pool)\n at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)\n at Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)\n at Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)\n at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)\n at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)\n at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection()\n at Hangfire.SqlServer.SqlServerConnection.AcquireLock(String resource, TimeSpan timeout)\n at Hangfire.Server.RecurringJobScheduler.UseConnectionDistributedLock[T](JobStorage storage, Func`2 action)\n at Hangfire.Server.RecurringJobScheduler.Execute(BackgroundProcessContext context)\n at Hangfire.Server.BackgroundProcessDispatcherBuilder.ExecuteProcess(Guid executionId, Object state)\n at Hangfire.Processing.BackgroundExecution.Run(Action`2 callback, Object state)\nClientConnectionId:00000000-0000-0000-0000-000000000000","State":"Execution RecurringJobScheduler is in the Failed state now due to an exception, execution will be retried no more than in 00:00:09","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":41,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:01:34.2558888Z","@mt":"{State:l}","@r":["Server be77b3cf-aa78-4693-a430-1d5f4756497b encountered an exception while sending heartbeat"],"@l":"Warning","@x":"Microsoft.Data.SqlClient.SqlException (0x80131904): Login failed for user 'sa'. Reason: Server is in script upgrade mode. Only administrator can connect at this time.\n at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)\n at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)\n at Microsoft.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)\n at Microsoft.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK)\n at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)\n at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance)\n at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken, DbConnectionPool pool)\n at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)\n at Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)\n at Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)\n at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)\n at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)\n at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection()\n at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func)\n at Hangfire.Server.ServerHeartbeatProcess.Execute(BackgroundProcessContext context)\nClientConnectionId:0e7e223c-40ad-4388-bb35-2ec4994b5610\nError Number:18401,State:1,Class:14","State":"Server be77b3cf-aa78-4693-a430-1d5f4756497b encountered an exception while sending heartbeat","SourceContext":"Hangfire.Server.ServerHeartbeatProcess","ThreadId":14,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:01:49.2052413Z","@mt":"{State:l}","@r":["Execution ServerWatchdog is in the Failed state now due to an exception, execution will be retried no more than in 00:00:25"],"@l":"Error","@x":"Microsoft.Data.SqlClient.SqlException (0x80131904): Login failed for user 'sa'. Reason: Server is in script upgrade mode. Only administrator can connect at this time.\n at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)\n at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)\n at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection()\n at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func)\n at Hangfire.Server.ServerWatchdog.Execute(BackgroundProcessContext context)\n at Hangfire.Server.BackgroundProcessDispatcherBuilder.ExecuteProcess(Guid executionId, Object state)\n at Hangfire.Processing.BackgroundExecution.Run(Action`2 callback, Object state)\nClientConnectionId:0e7e223c-40ad-4388-bb35-2ec4994b5610\nError Number:18401,State:1,Class:14","State":"Execution ServerWatchdog is in the Failed state now due to an exception, execution will be retried no more than in 00:00:25","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":15,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:01:49.2052413Z","@mt":"{State:l}","@r":["Execution ExpirationManager is in the Failed state now due to an exception, execution will be retried no more than in 00:00:25"],"@l":"Error","@x":"Microsoft.Data.SqlClient.SqlException (0x80131904): Login failed for user 'sa'. Reason: Server is in script upgrade mode. Only administrator can connect at this time.\n at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)\n at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)\n at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection()\n at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func)\n at Hangfire.SqlServer.ExpirationManager.UseConnectionDistributedLock(SqlServerStorage storage, Action`1 action)\n at Hangfire.SqlServer.ExpirationManager.CleanupTable(String query, String table, Int32 numberOfRecordsInSinglePass, CancellationToken cancellationToken, Action`1 additionalActions)\n at Hangfire.SqlServer.ExpirationManager.Execute(CancellationToken cancellationToken)\n at Hangfire.Processing.BackgroundExecution.Run(Action`2 callback, Object state)\nClientConnectionId:0e7e223c-40ad-4388-bb35-2ec4994b5610\nError Number:18401,State:1,Class:14","State":"Execution ExpirationManager is in the Failed state now due to an exception, execution will be retried no more than in 00:00:25","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":17,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:01:49.4623736Z","@mt":"{State:l}","@r":["Execution CountersAggregator is in the Failed state now due to an exception, execution will be retried no more than in 00:00:25"],"@l":"Error","@x":"Microsoft.Data.SqlClient.SqlException (0x80131904): Login failed for user 'sa'. Reason: Server is in script upgrade mode. Only administrator can connect at this time.\n at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)\n at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)\n at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)\n at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection()\n at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func)\n at Hangfire.SqlServer.CountersAggregator.Execute(CancellationToken cancellationToken)\n at Hangfire.Processing.BackgroundExecution.Run(Action`2 callback, Object state)\nClientConnectionId:0e7e223c-40ad-4388-bb35-2ec4994b5610\nError Number:18401,State:1,Class:14","State":"Execution CountersAggregator is in the Failed state now due to an exception, execution will be retried no more than in 00:00:25","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":18,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:02:04.4733494Z","@mt":"{State:l}","@r":["Server be77b3cf-aa78-4693-a430-1d5f4756497b is now able to continue sending heartbeats"],"State":"Server be77b3cf-aa78-4693-a430-1d5f4756497b is now able to continue sending heartbeats","SourceContext":"Hangfire.Server.ServerHeartbeatProcess","ThreadId":14,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:02:45.2275977Z","@mt":"{State:l}","@r":["Execution RecurringJobScheduler recovered from the Failed state after 00:01:41.0029515 and is in the Running state now"],"State":"Execution RecurringJobScheduler recovered from the Failed state after 00:01:41.0029515 and is in the Running state now","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":41,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:02:45.2325116Z","@mt":"{State:l}","@r":["Execution DelayedJobScheduler recovered from the Failed state after 00:01:41.0100443 and is in the Running state now"],"State":"Execution DelayedJobScheduler recovered from the Failed state after 00:01:41.0100443 and is in the Running state now","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":40,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:07:14.2087972Z","@mt":"{State:l}","@r":["Execution ServerWatchdog recovered from the Failed state after 00:05:25.0032381 and is in the Running state now"],"State":"Execution ServerWatchdog recovered from the Failed state after 00:05:25.0032381 and is in the Running state now","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":15,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:07:14.4713989Z","@mt":"{State:l}","@r":["Execution CountersAggregator recovered from the Failed state after 00:05:25.0086778 and is in the Running state now"],"State":"Execution CountersAggregator recovered from the Failed state after 00:05:25.0086778 and is in the Running state now","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":18,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"} {"@t":"2025-03-14T02:32:14.2196085Z","@mt":"{State:l}","@r":["Execution ExpirationManager recovered from the Failed state after 00:30:25.0137109 and is in the Running state now"],"State":"Execution ExpirationManager recovered from the Failed state after 00:30:25.0137109 and is in the Running state now","SourceContext":"Hangfire.Processing.BackgroundExecution","ThreadId":17,"EnvironmentName":"Production","ProcessId":4026476,"MachineName":"nc-ph-4626"}