> For the complete documentation index, see [llms.txt](https://docs.starrocks.io/llms.txt). This page is also available as Markdown at its `.md` URL.

# Troubleshooting Resource Isolation

This topic provides answers to some frequently asked questions about resource isolation.

## Resource Group[​](#resource-group "Direct link to Resource Group")

### What resources must be configured in a resource group?[​](#what-resources-must-be-configured-in-a-resource-group "Direct link to What resources must be configured in a resource group?")

The CPU resource limit must be configured. You must set either `cpu_weight` or `exclusive_cpu_core`, and the values must be greater than 0.

### Does StarRocks support hard resource limits?[​](#does-starrocks-support-hard-resource-limits "Direct link to Does StarRocks support hard resource limits?")

Yes. StarRocks supports hard limits for memory. From v3.3.5 onwards, StarRocks supports hard limits for CPU via `exclusive_cpu_cores`.

### How is CPU allocated among resource groups?[​](#how-is-cpu-allocated-among-resource-groups "Direct link to How is CPU allocated among resource groups?")

When multiple resource groups run queries simultaneously, CPU usage is proportional to each group’s `cpu_core_limit`. If the normal group exceeds the `BE vCPU cores - short_query.cpu_core_limit` within a scheduling cycle, it will not be scheduled further in that cycle.

### How are resources calculated for the `short_query` resource group?[​](#how-are-resources-calculated-for-the-short_query-resource-group "Direct link to how-are-resources-calculated-for-the-short_query-resource-group")

When the `short_query` resource group has running queries, the CPU limit of all normal groups becomes `BE vCPU cores − short_query.cpu_core_limit`. If the `short_query` resource group is idle, its resources can be used by normal groups.

### How are queries without a matched resource group handled?[​](#how-are-queries-without-a-matched-resource-group-handled "Direct link to How are queries without a matched resource group handled?")

They use the default resource group `default_wg`, which has the following resource limits and properties:

* `cpu_core_limit` = vCPU cores
* `mem_limit` = 100%
* `type` = `normal`

### If the resource group `rg3` has no queries and all resources are allocated to resource groups `rg1` and `rg2`, will those resources be reallocated when `rg3` receives a large query?[​](#if-the-resource-group-rg3-has-no-queries-and-all-resources-are-allocated-to-resource-groups-rg1-and-rg2-will-those-resources-be-reallocated-when-rg3-receives-a-large-query "Direct link to if-the-resource-group-rg3-has-no-queries-and-all-resources-are-allocated-to-resource-groups-rg1-and-rg2-will-those-resources-be-reallocated-when-rg3-receives-a-large-query")

Yes. Reclamation happens gradually and stabilizes within tens of milliseconds to seconds.

### What is the role of classifiers? What if no classifier matches, or two classifiers overlap?[​](#what-is-the-role-of-classifiers-what-if-no-classifier-matches-or-two-classifiers-overlap "Direct link to What is the role of classifiers? What if no classifier matches, or two classifiers overlap?")

If no classifier matches, the query falls back to the default resource group `default_wg`. Classifiers have weights; if a query matches multiple classifiers, the one with the highest weight is selected.

### Why do resource groups require `mem_limit` if BE already uses 90% memory by default?[​](#why-do-resource-groups-require-mem_limit-if-be-already-uses-90-memory-by-default "Direct link to why-do-resource-groups-require-mem_limit-if-be-already-uses-90-memory-by-default")

`mem_limit` restricts memory at the resource-group level. It applies only to queries match that resource group.

### If the resource group's `query_type` is set to `insert`, does INSERT INTO SELECT only limit INSERT or also SELECT?[​](#if-the-resource-groups-query_type-is-set-to-insert-does-insert-into-select-only-limit-insert-or-also-select "Direct link to if-the-resource-groups-query_type-is-set-to-insert-does-insert-into-select-only-limit-insert-or-also-select")

Only the SELECT part is limited by the resource group. The INSERT operation is not limited.

### If the resource groups `mem_limit` is set to `20%`, is the usable memory calculated as `BE_memory * 90% * 20%`? What if total `mem_limit` exceeds 100% when there are multiple resource groups?[​](#if-the-resource-groups-mem_limit-is-set-to-20-is-the-usable-memory-calculated-as-be_memory--90--20-what-if-total-mem_limit-exceeds-100-when-there-are-multiple-resource-groups "Direct link to if-the-resource-groups-mem_limit-is-set-to-20-is-the-usable-memory-calculated-as-be_memory--90--20-what-if-total-mem_limit-exceeds-100-when-there-are-multiple-resource-groups")

Total `mem_limit` across groups can exceed 100%. But if a query exceeds its resource group limit, it will fail.

### How can I verify whether a resource group is applied to a query?[​](#how-can-i-verify-whether-a-resource-group-is-applied-to-a-query "Direct link to How can I verify whether a resource group is applied to a query?")

Check `fe.audit.log` or run `EXPLAIN VERBOSE <SQL>` to view the matched resource group.

### Are resource groups defined per cluster or per BE node?[​](#are-resource-groups-defined-per-cluster-or-per-be-node "Direct link to Are resource groups defined per cluster or per BE node?")

Resources are divided per BE node, and the resource group configuration applies to all BE nodes in the cluster.

### How to inspect resource group usage or monitoring metrics?[​](#how-to-inspect-resource-group-usage-or-monitoring-metrics "Direct link to How to inspect resource group usage or monitoring metrics?")

Use FE/BE metrics endpoints to view specific resource-group-related metrics.

* For FE, collect the following metrics from `fe_host:8030/metrics?type=json`:

  * `starrocks_fe_query_resource_group`: The number of queries historically run in this resource group (including those currently running).
  * `starrocks_fe_query_resource_group_latency`: The query latency percentile for this resource group. The label type indicates specific percentiles, including `mean`, `75_quantile`, `95_quantile`, `98_quantile`, `99_quantile`, `999_quantile`.
  * `starrocks_fe_query_resource_group_err`: The number of queries in this resource group that encountered an error.

* For BE, collect the following metrics from `be_host:8040/metrics?type=json`:

  * `starrocks_be_resource_group_cpu_limit_ratio`: The ratio of this resource group's `cpu_core_limit` to the total `cpu_core_limit` across all resource groups.
  * `starrocks_be_resource_group_mem_limit_bytes`: The memory limit for this resource group.

### What is the difference between `short_query` and normal resource groups? Can I create multiple `short_query` resource groups?[​](#what-is-the-difference-between-short_query-and-normal-resource-groups-can-i-create-multiple-short_query-resource-groups "Direct link to what-is-the-difference-between-short_query-and-normal-resource-groups-can-i-create-multiple-short_query-resource-groups")

Only one `short_query` resource group is allowed. When there are queries running in `short_query` resource group, it uses actual BE cores, while normal groups share remaining resources proportionally.

### Does StarRocks provide query priority or large-query-based prioritization?[​](#does-starrocks-provide-query-priority-or-large-query-based-prioritization "Direct link to Does StarRocks provide query priority or large-query-based prioritization?")

No priority system exists. A query becomes a “large query” when exceeding any of the configured resource thresholds.

### Does a resource group belong to a specific BE node or the BE running the query?[​](#does-a-resource-group-belong-to-a-specific-be-node-or-the-be-running-the-query "Direct link to Does a resource group belong to a specific BE node or the BE running the query?")

Resource groups apply uniformly across all BEs in the cluster.

### How should I configure `concurrency_limit`?[​](#how-should-i-configure-concurrency_limit "Direct link to how-should-i-configure-concurrency_limit")

It depends on query complexity, cluster size, and workload patterns.

### How does classifier-based matching work? Is it tied to user and/or database?[​](#how-does-classifier-based-matching-work-is-it-tied-to-user-andor-database "Direct link to How does classifier-based matching work? Is it tied to user and/or database?")

Matching depends on classifier attributes such as IP, user, db, role, or `query_type`.

### How is resource group specified through session variables?[​](#how-is-resource-group-specified-through-session-variables "Direct link to How is resource group specified through session variables?")

You can set it as a variable:

```sql
SET resource_group = '<resource_group_name>';

```

Or specify it in queries via hint:

```sql
SELECT /*+ SET_VAR(SET resource_group = '<resource_group_name>') */ * FROM tbl;

```

### Does concurrency control take effect globally, per user, or per BE?[​](#does-concurrency-control-take-effect-globally-per-user-or-per-be "Direct link to Does concurrency control take effect globally, per user, or per BE?")

`concurrency_limit` restricts concurrency per resource group, while `pipeline_dop` controls parallelism of a single pipeline instance.

### Does memory limit take effect globally, per user, or per BE?[​](#does-memory-limit-take-effect-globally-per-user-or-per-be "Direct link to Does memory limit take effect globally, per user, or per BE?")

`mem_limit` applies to the resource group per BE. Per-instance memory is controlled by `exec_mem_limit`.

### Are concurrency and memory limits only effective when resource groups are enabled?[​](#are-concurrency-and-memory-limits-only-effective-when-resource-groups-are-enabled "Direct link to Are concurrency and memory limits only effective when resource groups are enabled?")

Concurrency is controlled only by resource groups. Query parallelism is controlled by session variables such as `pipeline_dop`.

### Will CTAS tasks match resource groups if the `query_type` is set to `INSERT`?[​](#will-ctas-tasks-match-resource-groups-if-the-query_type-is-set-to-insert "Direct link to will-ctas-tasks-match-resource-groups-if-the-query_type-is-set-to-insert")

Yes. Resource group will restrict the resources for the SELECT part of CTAS tasks. The big query limits will also take effect if the SELECT part exceeds the threshold.

### Why can't the queries in the `short_query` resource group consume all CPU?[​](#why-cant-the-queries-in-the-short_query-resource-group-consume-all-cpu "Direct link to why-cant-the-queries-in-the-short_query-resource-group-consume-all-cpu")

The `short_query` resource group must leave at least 1 CPU core for normal groups.

### Without query queue and resource groups, are concurrent queries limited?[​](#without-query-queue-and-resource-groups-are-concurrent-queries-limited "Direct link to Without query queue and resource groups, are concurrent queries limited?")

No. Overload results in query timeouts or memory limit errors.

### With resource groups enabled but query queue disabled, is concurrency limited by resource groups?[​](#with-resource-groups-enabled-but-query-queue-disabled-is-concurrency-limited-by-resource-groups "Direct link to With resource groups enabled but query queue disabled, is concurrency limited by resource groups?")

Yes. New queries exceeding the resource group `concurrency_limit` will fail.

### When is a query recognized as a "big query"?[​](#when-is-a-query-recognized-as-a-big-query "Direct link to When is a query recognized as a \"big query\"?")

A query is considered as a big query when it exceeds any of:

* `big_query_cpu_second_limit`
* `big_query_scan_rows_limit`
* `big_query_mem_limit`

### Can `default_wg` resource limits be changed?[​](#can-default_wg-resource-limits-be-changed "Direct link to can-default_wg-resource-limits-be-changed")

No. As a workaround, create a general resource group that can match all queries.

Example:

```sql
CREATE RESOURCE GROUP general_group TO (
    query_type IN ('select', 'insert') 
)
WITH (
    'cpu_core_limit' = '6', 
    'mem_limit' = '0.0000000000001'
);

```

### Why does the “query_resource_group” metric not show newly created groups?[​](#why-does-the-query_resource_group-metric-not-show-newly-created-groups "Direct link to Why does the “query_resource_group” metric not show newly created groups?")

The metric is lazy-initialized; it appears only after a query hits that group.

### If many queries run on a BE, and one hits CPU limit, do all fail?[​](#if-many-queries-run-on-a-be-and-one-hits-cpu-limit-do-all-fail "Direct link to If many queries run on a BE, and one hits CPU limit, do all fail?")

Only the query reaching the limit fails.

### If BE nodes have different memory/CPU sizes, does `mem_limit` or `cpu_core_limit` affect results?[​](#if-be-nodes-have-different-memorycpu-sizes-does-mem_limit-or-cpu_core_limit-affect-results "Direct link to if-be-nodes-have-different-memorycpu-sizes-does-mem_limit-or-cpu_core_limit-affect-results")

Memory limit is hard and may cause failures on BEs with less memory size first. CPU is soft and does not cause errors.

### Are `big_query_` parameters applied per node or globally?[​](#are-big_query_-parameters-applied-per-node-or-globally "Direct link to are-big_query_-parameters-applied-per-node-or-globally")

They apply per BE node.

### How to configure resource groups for Broker Load?[​](#how-to-configure-resource-groups-for-broker-load "Direct link to How to configure resource groups for Broker Load?")

Example classifier: `query_type="insert", user="alice"`.

### `exec_mem_limit`-related questions[​](#exec_mem_limit-related-questions "Direct link to exec_mem_limit-related-questions")

Q: How many instances will be generated for a query?

A: It is unpredictable because different queries can generate different number of fragments.

Q: How to check the number of instance?

A: You cannot check the number of instance.

Q: If a query consumes 128 GB memory in total and generates 60 instances, while `query_mem_limit=0` and `exec_mem_limit=2G`. Will the query fail?

A: The query fails as long as any instance consumes more than 2 GB (`exec_mem_limit`) memory.

### How to disable global query queue for one resource group only?[​](#how-to-disable-global-query-queue-for-one-resource-group-only "Direct link to How to disable global query queue for one resource group only?")

1. Enable resource group-level query queue:

   ```sql
   SET GLOBAL enable_show_all_variables = true;
   SET enable_group_level_query_queue = true;

   ```

2. Disable Query Queue for the current session or at user level:

   ```sql
   -- Disable Query Queue for the current session
   SET enable_query_queue = false;
   -- Disable Query Queue at user level
   ALTER USER 'xxx' SET PROPERTIES ("session.enable_query_queue" = "false");

   ```

### Can queries that doesn't match any resource group be forced to fail?[​](#can-queries-that-doesnt-match-any-resource-group-be-forced-to-fail "Direct link to Can queries that doesn't match any resource group be forced to fail?")

No. They will always fall back to the default resource group `default_wg`.

## Query Queue[​](#query-queue "Direct link to Query Queue")

### How is the query queue memory trigger calculated?[​](#how-is-the-query-queue-memory-trigger-calculated "Direct link to How is the query queue memory trigger calculated?")

Query Queue trigger = BE available memory size \* `query_queue_mem_used_pct_limit`.

### Which takes priority: resource-group concurrency or query-queue concurrency?[​](#which-takes-priority-resource-group-concurrency-or-query-queue-concurrency "Direct link to Which takes priority: resource-group concurrency or query-queue concurrency?")

If `enable_group_level_query_queue` is set to:

* `false`: global or group limit may trigger first.
* `true`: both apply; the smaller limit triggers queueing.

### If queue size or timeout is reached, will queries fail immediately?[​](#if-queue-size-or-timeout-is-reached-will-queries-fail-immediately "Direct link to If queue size or timeout is reached, will queries fail immediately?")

* When `query_queue_max_queued_queries` is reached, the query immediately fails.
* When `query_queue_concurrency_limit` is reached, the query waits in queue.

### What is the difference between resource group limits and query queue limits?[​](#what-is-the-difference-between-resource-group-limits-and-query-queue-limits "Direct link to What is the difference between resource group limits and query queue limits?")

Resource groups restrict resource usage per group per BE node.

Query queue uses BE-level limits for all queries.

`concurrency_limit` and `max_cpu_cores` both apply when resource group-level query queue is enabled.

### What is the difference between `pipeline_dop`, `exec_mem_limit`, and resource group concurrency limits?[​](#what-is-the-difference-between-pipeline_dop-exec_mem_limit-and-resource-group-concurrency-limits "Direct link to what-is-the-difference-between-pipeline_dop-exec_mem_limit-and-resource-group-concurrency-limits")

`pipeline_dop` controls in-query parallelism.

Resource groups/query queue control cluster-wide concurrent queries.

`query_mem_limit` controls per-query-per-BE memory.