How is your experience with the OutSystems Community?
37
Views
5
Comments
Solved by AI
Advanced SQL vs Aggregate for Single-Column Lookup on Exposed REST API

I have a simple lookup query used inside an exposed REST API endpoint, and I'm trying to determine the most performant approach between using an Aggregate vs an Advanced SQL node.

Context:

  • The query filters on a single indexed parameter (UserId, which has an index on the underlying table)
  • I only need one column back (USERCSN)
  • Max Records = 1 is sufficient — I only need the first matching row
  • This runs as part of a REST API that may be called frequently, so I want to minimize server-side processing time (TTFB)

Option 1 — Aggregate

Using the standard Aggregate node, by default it pulls back all attributes from the entity's output structure (ID, USERCSN, OSUSERID, IDTYPE, USERID) even though I only need USERCSN. To trim this down to a single column, do I need to enable Group By on the USERCSN attribute (essentially using it as a pseudo-DISTINCT), or is there a cleaner way to restrict the Aggregate's output to just one attribute without introducing a GROUP BY into the generated SQL?

My concern is that Group By forces SQL Server to perform an aggregation/sort step even when there's no actual aggregate function being used — which seems like unnecessary overhead for a query that's already narrowed down by an indexed WHERE clause and TOP 1.

Option 2 — Advanced SQL

Alternatively, I can just write:

SELECT TOP 1 {USEREXTENDEDTABLE}.[USERCSN] FROM {USEREXTENDEDTABLE}WHERE {USEREXTENDEDTABLE}.[USERID] = @UserId

This gives me exact control over the output (single column) and the generated execution plan, without relying on the Aggregate's auto-generated SQL.

Questions for the community:

  1. For a simple, single-table, single-output-column query like this, is there a meaningful performance difference between a properly-configured Aggregate and an Advanced SQL query?
  2. Is using Group By on a single attribute (as a way to force distinct/single-column output) considered a bad practice performance-wise, compared to just switching to Advanced SQL?
  3. Is there a way in the Aggregate editor to exclude specific attributes from the output structure without using Group By?
  4. For exposed REST APIs where TTFB matters, is Advanced SQL generally the recommended approach when the query is simple enough, or should Aggregate always be preferred for maintainability even at a slight performance cost?

Would appreciate insights from anyone who's benchmarked this or has strong opinions on Aggregate vs Advanced SQL best practices for lightweight, high-frequency lookup endpoints.

2021-09-06 15-09-53
Dorine Boudry
 
MVP
Solution

Hi @Don Den Cole Isaga ,

the mantra, regardless of whether it is for an exposed api or some other part of an application, is to go with aggregates unless you have a reason not to.  Just upfront deciding to use an sql widget because there might be performance reasons, leans towards premature optimization.  The time spent in the database and returning data from the database to the application is only a (small) part of total duration of the api call.

Without knowing your data, it's hard to help any further on this, there would be a real difference between an entity that has a total of 20 simple columns or an entity that has much more columns and some of them with large content.  Do you see a reason in entity structure to expect performance problems ?

This is something you could test in production(like) database, doing many calls with both methods and compare, more out of curiosity than need, i think.

Dorine

2026-05-01 02-54-23
Don Den Cole Isaga

Hi,

You have the same answer with Neo, our AI Assistant haha! I'll stick on using Aggregates not unless the logic requires very complex SQL queries that Aggregates cannot handle.

Thank you!

-DDC

2023-12-26 10-35-28
OutSystems Neo
Staff
Solution by AI
AI Generated

Short answer: stick with an Aggregate, don’t use Group By here, and only fall back to Advanced SQL if you truly hit limits of the Aggregate. The performance difference for this kind of lookup will be negligible if you configure the Aggregate correctly.

Let’s go through your points.


1. Aggregate vs Advanced SQL performance for this case

You have:

  • Single table
  • Filter on indexed column UserId
  • Max Records = 1
  • Need only one column (USERCSN)

Correctly configured, the Aggregate will generate a SELECT TOP 1 ... WHERE UserId = @UserId query that SQL Server will optimize very well. OutSystems aggregates are specifically optimized for this kind of usage and are generally preferred when they can express the query you need. Aggregates vs SQL > Performance Considerations for Data Fetching

Also, for “one record by indexed ID” scenarios, the best practice is explicitly to use Max. Records = 1 to avoid unnecessary data fetching. Aggregates Usage > Avoiding Unlimited Records

For such a simple query, the differences in TTFB between Aggregate and Advanced SQL will typically be in the noise compared to network latency and API stack overhead.


2. Using Group By just to reduce columns (pseudo‑DISTINCT)

Using Group By on a single attribute only to trim columns is not a good practice:

  • It conceptually misrepresents what the query is doing (you’re not aggregating; you’re just selecting).
  • It can force the database to perform grouping/sorting work that isn’t needed, especially if the engine decides it must sort/group to satisfy the GROUP BY. That’s extra overhead for no logical benefit in a “top 1 by indexed where” query.
  • It makes your Aggregate harder to read and maintain.

If you need DISTINCT semantics for a counting or grouping scenario, then Group By + aggregate functions is usually fine and is actually recommended over DISTINCT. Outsystems Group by vs SQL Distinct
But using Group By purely as a hack to reduce output columns is an anti-pattern. In that case, if you truly need custom projection, Advanced SQL is a cleaner option than abusing Group By.


3. Can you exclude attributes from the Aggregate output?

Not in the way you’re thinking.

Key points:

  • The Aggregate’s output structure is based on the entity, and you cannot selectively “turn off” attributes from the output in the editor.
  • OutSystems does optimize attribute fetching and output, even if the structure shows all fields. The engine can trim attributes when they’re not used downstream, and this is one of the reasons Aggregates are recommended. Aggregates or Simple Queries
  • In other words, seeing all attributes in the Aggregate result does not necessarily mean every column is materialized and transferred in a costly way if you never reference them.

So:

  • You can’t say “only fetch USERCSN” explicitly in the Aggregate UI.
  • But you can rely on OutSystems’ optimization to avoid paying a big penalty for unused attributes.
  • Do not use Group By as a workaround for this.

If you absolutely require a strictly minimal projection for some reason (e.g., very wide table, proven performance issue, or you want a special index hint / rowlock behavior), then Advanced SQL is the right tool—but that’s the exception, not the rule. How are aggregates better that advance SQL?


4. REST APIs, TTFB, and best practice choice

For exposed REST APIs:

  • Default rule: use Aggregates when they can express what you need.
    • They are easier to read, maintain, refactor, and they benefit from platform-level optimizations and automatic updates when entities change. Aggregates or Simple Queries
  • Advanced SQL: use it when:
    • You need features Aggregates don’t support (complex joins, subqueries, CTEs, unions, etc.).
    • You have very complex queries where Aggregates become unreadable or fragmented across multiple queries.
    • You need DML (INSERT/UPDATE/DELETE), or specific DB-level constructs. Aggregates or Simple Queries

In your particular case:

  • Indexed filter on UserId
  • Max Records = 1
  • Simple table lookup
  • No complex logic

The recommended approach is:

  1. Use an Aggregate.
  2. Set Max Records = 1.
  3. Filter by UserId as you are doing.
  4. Do not add Group By.

This will keep your logic in the OutSystems “happy path” and deliver performance that is effectively equivalent to your handwritten SELECT TOP 1 in most real-world scenarios.


Summary answers to your questions

  1. Performance difference Aggregate vs Advanced SQL here?
    For a single-table, indexed WHERE, Max Records = 1 lookup: typically no meaningful difference. Use an Aggregate.

  2. Is using Group By on a single attribute to force single-column output bad practice?
    Yes, it’s a misuse of Group By and can add unnecessary overhead. Prefer a plain Aggregate or, if you really need custom projection, Advanced SQL.

  3. Can you exclude specific attributes from Aggregate output without Group By?
    Not explicitly in the editor. The result structure always includes entity attributes, but the platform optimizes attribute fetching/output behind the scenes.

  4. For REST APIs with tight TTFB goals, is Advanced SQL generally recommended?
    No. Aggregates are generally preferred for maintainability and built-in optimizations. Use Advanced SQL when aggregates cannot express your query or when you need special DB-specific behavior.

If you later profile and find this specific lookup is a genuine hotspot, you can swap that one Aggregate for Advanced SQL with negligible refactor cost—but I’d start with the Aggregate approach.

This answer was AI-generated. Please read it carefully and use the forums for clarifications
2021-09-06 15-09-53
Dorine Boudry
 
MVP
Solution

Hi @Don Den Cole Isaga ,

the mantra, regardless of whether it is for an exposed api or some other part of an application, is to go with aggregates unless you have a reason not to.  Just upfront deciding to use an sql widget because there might be performance reasons, leans towards premature optimization.  The time spent in the database and returning data from the database to the application is only a (small) part of total duration of the api call.

Without knowing your data, it's hard to help any further on this, there would be a real difference between an entity that has a total of 20 simple columns or an entity that has much more columns and some of them with large content.  Do you see a reason in entity structure to expect performance problems ?

This is something you could test in production(like) database, doing many calls with both methods and compare, more out of curiosity than need, i think.

Dorine

2026-05-01 02-54-23
Don Den Cole Isaga

Hi,

You have the same answer with Neo, our AI Assistant haha! I'll stick on using Aggregates not unless the logic requires very complex SQL queries that Aggregates cannot handle.

Thank you!

-DDC

2023-12-26 10-35-28
OutSystems Neo
Staff
Solution by AI
AI Generated

Short answer: stick with an Aggregate, don’t use Group By here, and only fall back to Advanced SQL if you truly hit limits of the Aggregate. The performance difference for this kind of lookup will be negligible if you configure the Aggregate correctly.

Let’s go through your points.


1. Aggregate vs Advanced SQL performance for this case

You have:

  • Single table
  • Filter on indexed column UserId
  • Max Records = 1
  • Need only one column (USERCSN)

Correctly configured, the Aggregate will generate a SELECT TOP 1 ... WHERE UserId = @UserId query that SQL Server will optimize very well. OutSystems aggregates are specifically optimized for this kind of usage and are generally preferred when they can express the query you need. Aggregates vs SQL > Performance Considerations for Data Fetching

Also, for “one record by indexed ID” scenarios, the best practice is explicitly to use Max. Records = 1 to avoid unnecessary data fetching. Aggregates Usage > Avoiding Unlimited Records

For such a simple query, the differences in TTFB between Aggregate and Advanced SQL will typically be in the noise compared to network latency and API stack overhead.


2. Using Group By just to reduce columns (pseudo‑DISTINCT)

Using Group By on a single attribute only to trim columns is not a good practice:

  • It conceptually misrepresents what the query is doing (you’re not aggregating; you’re just selecting).
  • It can force the database to perform grouping/sorting work that isn’t needed, especially if the engine decides it must sort/group to satisfy the GROUP BY. That’s extra overhead for no logical benefit in a “top 1 by indexed where” query.
  • It makes your Aggregate harder to read and maintain.

If you need DISTINCT semantics for a counting or grouping scenario, then Group By + aggregate functions is usually fine and is actually recommended over DISTINCT. Outsystems Group by vs SQL Distinct
But using Group By purely as a hack to reduce output columns is an anti-pattern. In that case, if you truly need custom projection, Advanced SQL is a cleaner option than abusing Group By.


3. Can you exclude attributes from the Aggregate output?

Not in the way you’re thinking.

Key points:

  • The Aggregate’s output structure is based on the entity, and you cannot selectively “turn off” attributes from the output in the editor.
  • OutSystems does optimize attribute fetching and output, even if the structure shows all fields. The engine can trim attributes when they’re not used downstream, and this is one of the reasons Aggregates are recommended. Aggregates or Simple Queries
  • In other words, seeing all attributes in the Aggregate result does not necessarily mean every column is materialized and transferred in a costly way if you never reference them.

So:

  • You can’t say “only fetch USERCSN” explicitly in the Aggregate UI.
  • But you can rely on OutSystems’ optimization to avoid paying a big penalty for unused attributes.
  • Do not use Group By as a workaround for this.

If you absolutely require a strictly minimal projection for some reason (e.g., very wide table, proven performance issue, or you want a special index hint / rowlock behavior), then Advanced SQL is the right tool—but that’s the exception, not the rule. How are aggregates better that advance SQL?


4. REST APIs, TTFB, and best practice choice

For exposed REST APIs:

  • Default rule: use Aggregates when they can express what you need.
    • They are easier to read, maintain, refactor, and they benefit from platform-level optimizations and automatic updates when entities change. Aggregates or Simple Queries
  • Advanced SQL: use it when:
    • You need features Aggregates don’t support (complex joins, subqueries, CTEs, unions, etc.).
    • You have very complex queries where Aggregates become unreadable or fragmented across multiple queries.
    • You need DML (INSERT/UPDATE/DELETE), or specific DB-level constructs. Aggregates or Simple Queries

In your particular case:

  • Indexed filter on UserId
  • Max Records = 1
  • Simple table lookup
  • No complex logic

The recommended approach is:

  1. Use an Aggregate.
  2. Set Max Records = 1.
  3. Filter by UserId as you are doing.
  4. Do not add Group By.

This will keep your logic in the OutSystems “happy path” and deliver performance that is effectively equivalent to your handwritten SELECT TOP 1 in most real-world scenarios.


Summary answers to your questions

  1. Performance difference Aggregate vs Advanced SQL here?
    For a single-table, indexed WHERE, Max Records = 1 lookup: typically no meaningful difference. Use an Aggregate.

  2. Is using Group By on a single attribute to force single-column output bad practice?
    Yes, it’s a misuse of Group By and can add unnecessary overhead. Prefer a plain Aggregate or, if you really need custom projection, Advanced SQL.

  3. Can you exclude specific attributes from Aggregate output without Group By?
    Not explicitly in the editor. The result structure always includes entity attributes, but the platform optimizes attribute fetching/output behind the scenes.

  4. For REST APIs with tight TTFB goals, is Advanced SQL generally recommended?
    No. Aggregates are generally preferred for maintainability and built-in optimizations. Use Advanced SQL when aggregates cannot express your query or when you need special DB-specific behavior.

If you later profile and find this specific lookup is a genuine hotspot, you can swap that one Aggregate for Advanced SQL with negligible refactor cost—but I’d start with the Aggregate approach.

This answer was AI-generated. Please read it carefully and use the forums for clarifications
2020-09-15 13-07-23
Kilian Hekhuis
 
MVP

Hi Don,

Just one observation. You write "Using the standard Aggregate node, by default it pulls back all attributes from the entity's output structure". This is not true. Aggregates are optimized, and will only fetch the attributes that are in use. What is true, is that the output structure is kept, so if you only use, say, the Name attribute of an entity that has Id, Name and Date, the Aggregate returns 0, "The Name", NullDate(). But returning many attributes with a default value really does not take much time. The overhead of network, database engine and disk time is much, much larger. That's why Dorine already said to not use premature optimization.

UserImage.jpg
vikas sharma
Champion

Hi Don,

Aggregates are automatically optimized for the performance and it will only fetch the columns which are actively in use. For this you just need to avoid the isolated aggregates in any action. Please refer this for more information.


https://success.outsystems.com/documentation/11/building_apps/data_management/data_operations/best_practices_for_fetching_data/#Avoid_using_isolated_Aggregates


Community GuidelinesBe kind and respectful, give credit to the original source of content, and search for duplicates before posting.