OutSystems 11 Timer Fails on One Front-End Server

Back to Forums

IVIahdi

Question

IIS

JSON

Front-end

Development

Service Studio Version

11.55.93 (Build 65333)

Hi everyone,

I'm troubleshooting a recurring issue in an enterprise OutSystems 11 production environment and would appreciate some advice.

A timer consistently fails when its execution involves one specific front-end server, while executions on three other front-end servers succeed.

The issue has persisted for approximately four months. The infrastructure team has investigated but hasn't identified any issues and suggested looking into the application logic.

My questions:

1. What could cause a timer to fail on one front-end server but succeed on others?

2. How can I determine whether the issue is related to the server environment, application logic, or an internal HTTP request?

3. Which OutSystems logs or diagnostic tools would help identify the root cause?

Error details:

Message: The remote server returned an error: (500) Internal Server Error. Failed to parse the error response.

[1] The remote server returned an error: (500) Internal Server Error. Failed to parse the error response.

[2] Unexpected character encountered while parsing value: <. Path '', line 0, position 0.

Due to confidentiality requirements, I cannot share application names, server names, or other production details

Thanks for your help!

22 Oct

Akshay Deshpande

AI Generated

The error suggests that an HTTP request is returning a 500 response with a non-JSON body (likely HTML). The JSON parsing error is probably a secondary symptom, so I would first identify the original server-side error.

1. Possible causes

Front-end-specific issue: IIS/application pool configuration, corrupted ASP.NET temporary files, runtime differences, or connectivity problems.
Application logic: An unhandled exception, data-dependent failure, or timeout in the Timer's action.
Internal HTTP/service call: A consumed REST API, Service Action, or another module returning HTTP 500.
Infrastructure: Proxy/load balancer routing, DNS, authentication, or network issues affecting that node.

2. How to isolate the root cause

In Service Center → Monitoring → Timers, identify the failed execution and check the Server, Error, execution time, and timestamp.
Compare the same Timer execution on the failing and successful front-end servers.
Check whether the Timer fails before entering its main logic or at a specific integration/action.
If an HTTP or Service Action is involved, inspect its response status, response body, request URL, and corresponding server-side logs.
Compare the failing server's IIS configuration, application pool, network connectivity, and deployed runtime files with the working servers.

3. Logs and diagnostic tools

Service Center → Monitoring → Errors: Find the original exception and stack trace.
Service Center → Monitoring → Timers: Identify the executing front-end server and failed execution.
Service Center → Monitoring → Integrations / Service Actions: Investigate consumed APIs and service calls.
IIS logs and Windows Event Viewer: Check for HTTP 500 details, application pool failures, and runtime errors.
OutSystems Scheduler Service logs and Environment Health: Investigate scheduler threads and front-end health.

References:

My recommendation: Since the failure consistently correlates with one front-end server, start by correlating the Timer's Executed By value with the original error and IIS logs. This is strong evidence to investigate that node, but it does not rule out application logic or an HTTP call that behaves differently on that server. Avoid restarting services or changing production configuration until the relevant logs have been captured.

Thanks and Regards,
Akshay Deshpande

23 Oct

This answer was AI-generated. Please read it carefully and use the forums for clarifications

Ahmed Essawy

The two errors together are the key clue. “(500)… Failed to parse the error response” plus “Unexpected character ‘<’” means the timer’s logic called something over HTTP, and it answered with an HTML error page instead of JSON.

In O11 that “something” is often not an external API. It is frequently a Service Action or an exposed REST method in another module, because Service Actions are HTTP calls under the hood. So the timer itself is probably fine. The call target fails when the call involves that front-end.

How to pin it down:

Find the failing call. In Service Center > Monitoring > Errors, open the error and check the stack and the action. Then check Monitoring > Integrations at the same timestamp to see the URL and the server.
Get the real error behind the 500. Look for an error logged by the called module in the same second. If nothing is there, the request never reached OutSystems code. In that case, check the IIS logs on that front-end for that URL (status 500 and its sub-status) and the Windows Event Viewer (Application log).
Reproduce locally. RDP into the bad front-end and open the target URL in a browser there. Locally, ASP.NET shows the detailed error page, which usually states the root cause in plain text.

Typical causes when only one node fails:

The producer module is not deployed correctly on that server. Check Environment Health; republishing the producer often fixes it.
A proxy is configured only on that server (machine.config, WinHTTP, or environment variables).
Hosts file or DNS differences.
TLS or certificate trust differences.
Windows/.NET updates or app pool settings that differ from the other three nodes.

After four months, comparing that node with a healthy one on these points will likely be faster than reviewing the timer logic.

14 days ago

Community GuidelinesBe kind and respectful, give credit to the original source of content, and search for duplicates before posting.

See the full guidelines