Edit
Feedback

Parameter configuration

This topic describes FE, BE, and system parameters. It also provides suggestions on how to configure and tune these parameters.

FE configuration items

FE parameters are classified into dynamic parameters and static parameters.

  • Dynamic parameters can be configured and adjusted by running SQL commands, which is very convenient. But the configurations become invalid after you restart your FE.

  • Static parameters can only be configured and adjusted in the FE configuration file fe.conf. After you modify this file, you must restart your FE for the changes to take effect.

Whether a parameter is a dynamic parameter is indicated by the IsMutable column in the output of ADMIN SHOW CONFIG. TRUE indicates a dynamic parameter.

Note that both dynamic and static FE parameters can be configured in the fe.conf file.

View FE configuration items

After your FE is started, you can run the ADMIN SHOW FRONTEND CONFIG command on your MySQL client to check the parameter configurations. If you want to query the configuration of a specific parameter, run the following command:

 ADMIN SHOW FRONTEND CONFIG [LIKE "pattern"];

For detailed description of the returned fields, see ADMIN SHOW CONFIG.

NOTE

You must have administrator's privilege to run cluster administration-related commands.

Configure FE dynamic parameters

You can configure or modify the settings of FE dynamic parameters using ADMIN SET FRONTEND CONFIG.

ADMIN SET FRONTEND CONFIG ("key" = "value");

Logging

ParameterUnitDefaultDescription
qe_slow_log_msms5000The threshold used to determine whether a query is a slow query. If the response time of a query exceeds this threshold, it is recorded as a slow query in fe.audit.log.

Metadata and cluster management

ParameterUnitDefaultDescription
catalog_try_lock_timeout_msms5000The timeout duration to obtain the global lock.
edit_log_roll_num-50000The maximum number of metadata log entries that can be written before a log file is created for these log entries.
This parameter is used to control the size of log files. The new log file is written to the BDBJE database.
ignore_unknown_log_id-FALSEWhether to ignore an unknown log ID. When an FE is rolled back, the BEs of the earlier version may be unable to recognize some log IDs.
If the value is TRUE, the FE ignores unknown log IDs. If the value is FALSE, the FE exits.
ignore_meta_check-FALSEWhether non-leader FEs ignore the metadata gap from the leader FE. If the value is TRUE, non-leader FEs ignore the metadata gap from the leader FE and continues providing data reading services.
This parameter ensures continuous data reading services even when you stop the leader FE for a long period of time.
If the value is FALSE, non-leader FEs do not ignore the metadata gap from the leader FE and stop providing data reading services.
meta_delay_toleration_seconds300The maximum duration by which the metadata on the follower and observer FEs can lag behind that on the leader FE. Unit: seconds.
If this duration is exceeded, the non-leader FE stops providing services.
drop_backend_after_decommission-TRUEWhether to delete a BE after the BE is decommissioned. TRUE indicates that the BE is deleted immediately after it is decommissioned.
FALSE indicates that the BE is not deleted after it is decommissioned.
enable_collect_query_detail_info-FALSEWhether to view the profile of a query. If this parameter is set to TRUE, the system collects the profile of the query.
If this parameter is set to FALSE, the system does not collect the profile of the query.

Query engine

ParameterUnitDefaultDescription
max_allowed_in_element_num_of_delete-10000The maximum number of elements allowed for the IN predicate in a DELETE statement.
enable_materialized_view-TRUEWhether to enable the creation of materialized views.
enable_decimal_v3-TRUEWhether to support the DECIMAL V3 data type.
enable_sql_blacklist-FALSEWhether to enable blacklist check for SQL queries. When this feature is enabled, queries in the blacklist cannot be executed.
dynamic_partition_check_interval_secondss600The interval at which new data is checked. If new data is detected, StarRocks automatically creates partitions for the data.
dynamic_partition_enable-TRUEWhether to enable the dynamic partitioning feature. When this feature is enabled, StarRocks dynamically creates partitions for new data and automatically deletes expired partitions to ensure the freshness of data.
max_partitions_in_one_batch-4096The maximum number of partitions that can be created when you bulk create partitions.
max_query_retry_time-2The maximum number of query retries on an FE.
max_create_table_timeout_seconds600The maximum timeout duration for creating a table, in seconds.
max_running_rollup_job_num_per_table-1The maximum number of rollup jobs can run in parallel for a table.
max_planner_scalar_rewrite_num-100000The maximum number of times that the optimizer can rewrite a scalar operator.
enable_statistic_collect-TRUEWhether to collect statistics for the CBO. This feature is enabled by default.
enable_collect_full_statistic-TRUEWhether to enable automatic full statistics collection. This feature is enabled by default.
statistic_auto_collect_ratio-0.8The threshold for determining whether the statistics for automatic collection are healthy. If statistics health is below this threshold, automatic collection is triggered.
statistic_max_full_collect_data_sizeGB100The size of the largest partition for automatic collection to collect data. Unit: GB.If a partition exceeds this value, full collection is discarded and sampled collection is performed instead.
statistic_collect_interval_secs300The interval for checking data updates during automatic collection. Unit: seconds.
statistic_auto_analyze_start_timeSTRING00:00:00The start time of automatic collection. Value range: 00:00:00 - 23:59:59.
statistic_auto_analyze_end_timeSTRING23:59:59The end time of automatic collection. Value range: 00:00:00 - 23:59:59.
statistic_sample_collect_rows-200000The minimum number of rows to collect for sampled collection. If the parameter value exceeds the actual number of rows in your table, full collection is performed.
histogram_buckets_size-64The default bucket number for a histogram.
histogram_mcv_size-100The number of most common values (MCV) for a histogram.
histogram_sample_ratio-0.1The sampling ratio for a histogram.
histogram_max_sample_row_count-10000000The maximum number of rows to collect for a histogram.
statistics_manager_sleep_time_secs60The interval at which metadata is scheduled. Unit: seconds. The system performs the following operations based on this interval:
  • Create tables for storing statistics.
  • Delete statistics that have been deleted.
  • Delete expired statistics.
statistic_update_interval_secs24 * 60 * 60The interval at which the cache of statistical information is updated. Unit: seconds.
statistic_analyze_status_keep_seconds259200The duration to retain the history of collection tasks. The default value is 3 days. Unit: seconds.
statistic_collect_concurrency-3The maximum number of manual collection tasks that can run in parallel. The value defaults to 3, which means you can run a maximum of three manual collections tasks in parallel.
If the value is exceeded, incoming tasks will be in the PENDING state, waiting to be scheduled.
enable_local_replica_selection-FALSEWhether to select local replicas for queries. Local replicas reduce the network transmission cost. If this parameter is set to TRUE, the CBO preferentially selects tablet replicas on BEs that have the same IP address as the current FE. If this parameter is set to FALSE, both local replicas and non-local replicas can be selected. The default value is FALSE.
max_distribution_pruner_recursion_depth-100The maximum recursion depth allowed by the partition pruner. Increasing the recursion depth can prune more elements but also increases CPU consumption.
enable_udf-FALSEWhether to enable UDF .

Loading and unloading

ParameterUnitDefaultDescription
load_straggler_wait_seconds300The maximum loading lag that can be tolerated by a BE replica. If this value is exceeded, cloning is performed to clone data from other replicas. Unit: seconds.
desired_max_waiting_jobs-100The maximum number of pending jobs in an FE. The number refers to all jobs, such as table creation, loading, and schema change jobs. If the number of pending jobs in an FE reaches this value, the FE will reject new load requests. This parameter takes effect only for asynchronous loading.
max_load_timeout_seconds259200The maximum timeout duration allowed for a load job. The load job fails if this limit is exceeded. This limit applies to all types of load jobs. Unit: seconds.
min_load_timeout_seconds1The minimum timeout duration allowed for a load job. This limit applies to all types of load jobs. Unit: seconds.
max_running_txn_num_per_db-100The maximum number of load jobs that can run in parallel for each database in a StarRocks cluster. The default value is 100. If this value is exceeded, incoming load jobs will not be executed. If the incoming load job is a synchronous load job, it will be rejected. If it is an asynchronous load job, it will be put into the waiting queue. We do not recommend you increase this value because this will increase system load.
load_parallel_instance_num-1The maximum number of concurrent loading instances for each load job on a BE.
disable_load_job-FALSEWhether to disable loading when the cluster encounters an error. This prevents any loss caused by cluster errors. The default value is FALSE, indicating that loading is not disabled.
history_job_keep_max_seconds604800The maximum duration a historical job can be retained, such as schema change jobs, in seconds.
label_keep_max_num-1000The maximum number of load jobs that can be retained within a period of time. If this number is exceeded, the information of historical jobs will be deleted.
label_keep_max_seconds259200The maximum duration the labels of load jobs that have been completed and are in the FINISHED or CANCELLED state can be retained in the StarRocks system. The default value is 3 days. After this duration expires, the labels will be deleted. This parameter applies to all types of load jobs. Unit: seconds. A value too large consumes a lot of memory.
max_routine_load_job_num-100The maximum number of Routine Load jobs in a StarRocks cluster.
max_routine_load_task_concurrent_num-5The maximum number of concurrent tasks for each Routine Load job.
max_routine_load_task_num_per_be-5The maximum number of concurrent Routine Load tasks that can run for each BE. The value must be less than or equal to the BE configuration item routine_load_thread_pool_size.
max_routine_load_batch_sizeByte4294967296The maximum amount of data that can be loaded by a Routine Load task, in bytes.
routine_load_task_consume_seconds15The maximum duration each Routine Load task can consume data, in seconds.
routine_load_task_timeout_seconds60The timeout duration for each Routine Load task, in seconds.
max_tolerable_backend_down_num-0The maximum number of faulty BE nodes allowed. If this number is exceeded, Routine Load jobs cannot be automatically recovered.
period_of_auto_resume_minMin5The interval at which Routine Load jobs are automatically recovered, in minutes.
spark_load_default_timeout_seconds86400The timeout duration for each Spark Load job, in seconds.
spark_home_default_dir-StarRocksFE.STARROCKS_HOME_DIR + "/lib/spark2x"The root directory of a Spark client.
stream_load_default_timeout_seconds600The default timeout duration for each Stream Load job, in seconds.
max_stream_load_timeout_seconds259200The maximum allowed timeout duration for a Stream Load job, in seconds.
insert_load_default_timeout_seconds3600The timeout duration for the INSERT INTO statement that is used to load data, in seconds.
broker_load_default_timeout_seconds14400The timeout duration for a Broker Load job, in seconds.
min_bytes_per_broker_scannerByte67108864The minimum allowed amount of data that can be processed by a Broker Load instance, in bytes.
max_broker_concurrency-100The maximum number of concurrent instances for a Broker Load task.
export_max_bytes_per_be_per_taskByte268435456The maximum amount of data that can be exported from a single BE by a single data unload task, in bytes.
export_running_job_num_limit-5The maximum number of data exporting tasks that can run in parallel.
export_task_default_timeout_seconds7200The timeout duration for a data exporting task, in seconds.
empty_load_as_error-TRUEWhether to return an error message "all partitions have no load data" if no data is loaded. Values:
- TRUE: If no data is loaded, the system displays a failure message and returns an error "all partitions have no load data".
- FALSE: If no data is loaded, the system displays a success message and returns OK, instead of an error.

Storage

ParameterUnitDefaultDescription
enable_strict_storage_medium_check-FALSEWhether the FE strictly checks the storage medium of BEs when users create tables. If this parameter is set to TRUE, the FE checks the storage medium of BEs when users create tables and returns an error if the storage medium of the BE is different from the storage_medium parameter specified in the CREATE TABLE statement. For example, the storage medium specified in the CREATE TABLE statement is SSD but the actual storage medium of BEs is HDD. As a result, the table creation fails. If this parameter is FALSE, the FE does not check the storage medium of BEs when users create table.
capacity_used_percent_high_water-0.75The upper limit of disk usage on a BE. If this value is exceeded, table creation or clone jobs will not be sent to this BE, until the disk usage returns to normal.
storage_high_watermark_usage_percent%85The upper limit of storage space usage for BE's storage directory. If this value is exceeded, data can no longer be stored in this storage path.
storage_min_left_capacity_bytesByte2 * 1024 * 1024 * 1024The minimum remaining storage space allowed in the BE storage directory, in Bytes. If this value is exceeded, data can no longer be stored in this storage path.
catalog_trash_expire_seconds86400The longest duration the metadata can be retained after a table or database is deleted. If this duration expires, the data will be deleted and cannot be recovered. Unit: seconds.
alter_table_timeout_seconds86400The timeout duration for the schema change operation (ALTER TABLE). Unit: seconds.
recover_with_empty_tablet-FALSEWhether to replace a lost or corrupted tablet replica with an empty one. If a tablet replica is lost or corrupted, data queries on this tablet or other healthy tablets may fail. Replacing the lost or corrupted tablet replica with an empty tablet ensures that the query can still be executed. However, the result may be incorrect because data is lost. The default value is FALSE, which means lost or corrupted tablet replicas are not replaced with empty ones and the query fails.
tablet_create_timeout_seconds1The timeout duration for creating a tablet, in seconds.
tablet_delete_timeout_seconds2The timeout duration for deleting a tablet, in seconds.
check_consistency_default_timeout_seconds600The timeout duration for a replica consistency check. You can set this parameter based on the size of your tablet.
tablet_sched_slot_num_per_path-2The maximum number of tablet-related tasks that can run concurrently in a BE storage directory. The alias is schedule_slot_num_per_path.
tablet_sched_max_scheduling_tablets-2000The maximum number of tablets that can be scheduled at the same time. If the value is exceeded, tablet balancing and repair checks will be skipped.
tablet_sched_disable_balance-FALSEWhether to disable tablet balancing. TRUE indicates that tablet balancing is disabled. FALSE indicates that tablet balancing is enabled. The alias is disable_balance.
tablet_sched_disable_colocate_balance-FALSEWhether to disable replica balancing for Colocate Table. TRUE indicates replica balancing is disabled. FALSE indicates replica balancing is enabled. The alias is disable_colocate_balance.
tablet_sched_max_balancing_tablets-100The maximum number of tablets that can be balanced at the same time. If this value is exceeded, tablet re-balancing will be skipped. The alias is max_balancing_tablets.
tablet_sched_balance_load_disk_safe_threshold-0.5The threshold for determining whether the BE disk usage is balanced. This parameter takes effect only when tablet_sched_balancer_strategy is set to disk_and_tablet. If the disk usage of all BEs is lower than 50%, disk usage is considered balanced. For the disk_and_tablet policy, if the difference between the highest and lowest BE disk usage is greater than 10%, disk usage is considered unbalanced and tablet re-balancing is triggered. The alias is balance_load_disk_safe_threshold.
tablet_sched_balance_load_score_threshold-0.1The threshold for determining whether the BE load is balanced. This parameter takes effect only when tablet_sched_balancer_strategy is set to be_load_score. A BE whose load is 10% lower than the average load is in low load state, and a BE whose load is 10% higher than the average load is in high load state. The alias is balance_load_score_threshold.
tablet_sched_repair_delay_factor_seconds60The interval at which replicas are repaired, in seconds. The alias is tablet_repair_delay_factor_second.
tablet_sched_min_clone_task_timeout_secs3 * 60The minimum timeout duration for cloning a tablet, in seconds.
tablet_sched_max_clone_task_timeout_secs2 * 60 * 60The maximum timeout duration for cloning a tablet, in seconds. The alias is max_clone_task_timeout_sec.

Other FE dynamic parameters

ParameterUnitDefaultDescription
plugin_enable-TRUEWhether plugins can be installed on FEs. Plugins can be installed or uninstalled only on the Leader FE.
max_small_file_number-100The maximum number of small files that can be stored on an FE directory.
max_small_file_size_bytesByte1024 * 1024The maximum size of a small file, in bytes.
agent_task_resend_wait_time_msms5000The duration the FE must wait before it can resend an agent task. An agent task can be resent only when the gap between the task creation time and the current time exceeds the value of this parameter. This parameter is used to prevent repetitive sending of agent tasks. Unit: ms.
backup_job_default_timeout_msms86400*1000The timeout duration of a backup job, in ms. If this value is exceeded, the backup job fails.
report_queue_size-100The maximum number of jobs that can wait in a report queue.
The report is about disk, task, and tablet information of BEs. If too many report jobs are piling up in a queue, OOM will occur.
enable_experimental_mv-FALSEWhether to enable the asynchronous materialized view feature. TRUE indicates this feature is enabled.
authentication_ldap_simple_bind_base_dn-Empty stringThe base DN, which is the point from which the LDAP server starts to search for users' authentication information.
authentication_ldap_simple_bind_root_dn-Empty stringThe administrator DN used to search for users' authentication information.
authentication_ldap_simple_bind_root_pwd-Empty stringThe password of the administrator used to search for users' authentication information.
authentication_ldap_simple_server_host-Empty stringThe host on which the LDAP server runs.
authentication_ldap_simple_server_port-389The port of the LDAP server.
authentication_ldap_simple_user_search_attr-uidThe name of the attribute that identifies users in LDAP objects.

Configure FE static parameters

This section provides an overview of the static parameters that you can configure in the FE configuration file fe.conf. After you reconfigure these parameters for an FE, you must restart the FE for the changes to take effect.

Logging

ParameterDefaultDescription
log_roll_size_mb1024The size per log file. Unit: MB. The default value 1024 specifies the size per log file as 1 GB.
sys_log_dirStarRocksFE.STARROCKS_HOME_DIR + "/log"The directory that stores system log files.
sys_log_levelINFOThe severity levels into which system log entries are classified. Valid values: INFO, WARN, ERROR, and FATAL.
sys_log_verbose_modulesEmpty stringThe modules for which StarRocks generates system logs. If this parameter is set to org.apache.starrocks.catalog, StarRocks generates system logs only for the catalog module.
sys_log_roll_intervalDAYThe time interval at which StarRocks rotates system log entries. Valid values: DAY and HOUR.
  • If this parameter is set to DAY, a suffix in the yyyyMMdd format is added to the names of system log files.
  • If this parameter is set to HOUR, a suffix in the yyyyMMddHH format is added to the names of system log files.
sys_log_delete_age7dThe retention period of system log files. The default value 7d specifies that each system log file can be retained for 7 days. StarRocks checks each system log file and deletes those that were generated 7 days ago.
sys_log_roll_num10The maximum number of system log files that can be retained within each retention period specified by the sys_log_roll_interval parameter.
audit_log_dirStarRocksFE.STARROCKS_HOME_DIR + "/log"The directory that stores audit log files.
audit_log_roll_num90The maximum number of audit log files that can be retained within each retention period specified by the audit_log_roll_interval parameter.
audit_log_modulesslow_query, queryThe modules for which StarRocks generates audit log entries. By default, StarRocks generates audit logs for the slow_query module and the query module. Separate the module names with a comma (,) and a space.
audit_log_roll_intervalDAYThe time interval at which StarRocks rotates audit log entries. Valid values: DAY and HOUR.
  • If this parameter is set to DAY, a suffix in the yyyyMMdd format is added to the names of audit log files.
  • If this parameter is set to HOUR, a suffix in the yyyyMMddHH format is added to the names of audit log files.
audit_log_delete_age30dThe retention period of audit log files. The default value 30d specifies that each audit log file can be retained for 30 days. StarRocks checks each audit log file and deletes those that were generated 30 days ago.
dump_log_dirStarRocksFE.STARROCKS_HOME_DIR + "/log"The directory that stores dump log files.
dump_log_modulesqueryThe modules for which StarRocks generates dump log entries. By default, StarRocks generates dump logs for the the query module. Separate the module names with a comma (,) and a space.
dump_log_roll_intervalDAYThe time interval at which StarRocks rotates dump log entries. Valid values: DAY and HOUR.
  • If this parameter is set to DAY, a suffix in the yyyyMMdd format is added to the names of dump log files.
  • If this parameter is set to HOUR, a suffix in the yyyyMMddHH format is added to the names of dump log files.
dump_log_roll_num10The maximum number of dump log files that can be retained within each retention period specified by the dump_log_roll_interval parameter.
dump_log_delete_age7dThe retention period of dump log files. The default value 7d specifies that each dump log file can be retained for 7 days. StarRocks checks each dump log file and deletes those that were generated 7 days ago.

Server

ParameterDefaultDescription
frontend_address0.0.0.0The IP address of the FE node.
priority_networksEmpty stringDeclares a selection strategy for servers that have multiple IP addresses. Note that at most one IP address must match the list specified by this parameter. The value of this parameter is a list that consists of entries, which are separated with semicolons (;) in CIDR notation, such as 10.10.10.0/24. If no IP address matches the entries in this list, an IP address will be randomly selected.
http_port8030The port on which the HTTP server in the FE node listens.
http_backlog_num1024The length of the backlog queue held by the HTTP server in the FE node.
cluster_nameStarRocks ClusterThe name of the StarRocks cluster to which the FE belongs. The cluster name is displayed for Title on the web page.
rpc_port9020The port on which the Thrift server in the FE node listens.
thrift_backlog_num1024The length of the backlog queue held by the Thrift server in the FE node.
thrift_server_typeTHREAD_POOLThe service model that is used by the Thrift server in the FE node. Valid values: SIMPLE, THREADED, and THREAD_POOL.
thrift_server_max_worker_threads4096The maximum number of worker threads that are supported by the Thrift server in the FE node.
thrift_client_timeout_ms5000The length of time after which idle client connections time out. Unit: ms.
thrift_server_queue_size4096The length of queue where requests are pending. If the number of threads that are being processed in the thrift server exceeds the value specified in thrift_server_max_worker_threads, new requests are added to the pending queue.
brpc_idle_wait_max_time10000The maximum length of time for which BRPC clients wait as in the idle state. Unit: ms.
query_port9030The port on which the MySQL server in the FE node listens.
mysql_service_nio_enabledTRUESpecifies whether asynchronous I/O is enabled for the FE node.
mysql_service_io_threads_num4The maximum number of threads that can be run by the MySQL server in the FE node to process I/O events.
mysql_nio_backlog_num1024The length of the backlog queue held by the MySQL server in the FE node.
max_mysql_service_task_threads_num4096The maximum number of threads that can be run by the MySQL server in the FE node to process tasks.
max_connection_scheduler_threads_num4096The maximum number of threads that are supported by the connection scheduler.
qe_max_connection1024The maximum number of connections that can be established by all users to the FE node.
check_java_versionTRUESpecifies whether to check version compatibility between the executed and compiled Java programs. If the versions are incompatible, StarRocks reports errors and aborts the startup of Java programs.

Metadata and cluster management

ParameterDefaultDescription
meta_dirStarRocksFE.STARROCKS_HOME_DIR + "/meta"The directory that stores metadata.
heartbeat_mgr_threads_num8The number of threads that can be run by the Heartbeat Manager to run heartbeat tasks.
heartbeat_mgr_blocking_queue_size1024The size of the blocking queue that stores heartbeat tasks run by the Heartbeat Manager.
metadata_failure_recoveryFALSESpecifies whether to forcibly reset the metadata of the FE. Exercise caution when you set this parameter.
edit_log_port9010The port that is used for communication among the leader, follower, and observer FEs in the StarRocks cluster.
edit_log_typeBDBThe type of edit log that can be generated. Set the value to BDB.
bdbje_heartbeat_timeout_second30The amount of time after which the heartbeats among the leader, follower, and observer FEs in the StarRocks cluster time out. Unit: second.
bdbje_lock_timeout_second1The amount of time after which a lock in the BDB JE-based FE times out. Unit: second.
max_bdbje_clock_delta_ms5000The maximum clock offset that is allowed between the leader FE and the follower or observer FEs in the StarRocks cluster. Unit: ms.
txn_rollback_limit100The maximum number of transactions that can be rolled back.
bdbje_replica_ack_timeout_second10The maximum amount of time for which the leader FE can wait for ACK messages from a specified number of follower FEs when metadata is written from the leader FE to the follower FEs. Unit: second. If a large amount of metadata is being written, the follower FEs require a long time before they can return ACK messages to the leader FE, causing ACK timeout. In this situation, metadata writes fail, and the FE process exits. We recommend that you increase the value of this parameter to prevent this situation.
master_sync_policySYNCThe policy based on which the leader FE flushes logs to disk. This parameter is valid only when the current FE is a leader FE. Valid values:
  • SYNC: When a transaction is committed, a log entry is generated and flushed to disk simultaneously.
  • NO_SYNC: The generation and flushing of a log entry do not occur at the same time when a transaction is committed.
  • WRITE_NO_SYNC: When a transaction is commited, a log entry is generated simultaneously but is not flushed to disk.
If you have deployed only one follower FE, we recommend that you set this parameter to SYNC. If you have deployed three or more follower FEs, we recommend that you set this parameter and the replica_sync_policy both to WRITE_NO_SYNC.
replica_sync_policySYNCThe policy based on which the follower FE flushes logs to disk. This parameter is valid only when the current FE is a follower FE. Valid values:
  • SYNC: When a transaction is committed, a log entry is generated and flushed to disk simultaneously.
  • NO_SYNC: The generation and flushing of a log entry do not occur at the same time when a transaction is committed.
  • WRITE_NO_SYNC: When a transaction is committed, a log entry is generated simultaneously but is not flushed to disk.
replica_ack_policySIMPLE_MAJORITYThe policy based on which a log entry is considered valid. The default value SIMPLE_MAJORITY specifies that a log entry is considered valid if a majority of follower FEs return ACK messages.
cluster_id-1The ID of the StarRocks cluster to which the FE belongs. FEs or BEs that have the same cluster ID belong to the same StarRocks cluster. Valid values: any positive integer. The default value -1 specifies that StarRocks will generate a random cluster ID for the StarRocks cluster at the time when the leader FE of the cluster is started for the first time.

Query engine

ParameterDefaultDescription
publish_version_interval_ms10The time interval at which release validation tasks are issued. Unit: ms.
statistic_cache_columns100000The number of rows that can be cached for the statistics table.

Loading and unloading

ParameterDefaultDescription
async_load_task_pool_size10The size of the load task thread pool. This parameter is valid only for Broker Load.
load_checker_interval_second5The time interval at which load jobs are processed on a rolling basis. Unit: second.
transaction_clean_interval_second30The time interval at which finished transactions are cleaned up. Unit: second. We recommend that you specify a short time interval to ensure that finished transactions can be cleaned up in a timely manner.
label_clean_interval_second14400The time interval at which labels are cleaned up. Unit: second. We recommend that you specify a short time interval to ensure that historical labels can be cleaned up in a timely manner.
spark_dpp_version1.0.0The version of Spark Dynamic Partition Pruning (DPP) used.
spark_resource_pathEmpty stringThe root directory of the Spark dependency package.
spark_launcher_log_dirsys_log_dir + "/spark_launcher_log"The directory that stores Spark log files.
yarn_client_pathStarRocksFE.STARROCKS_HOME_DIR + "/lib/yarn-client/hadoop/bin/yarn"The root directory of the Yarn client package.
yarn_config_dirStarRocksFE.STARROCKS_HOME_DIR + "/lib/yarn-config"The directory that stores the Yarn configuration file.
export_checker_interval_second5The time interval at which load jobs are scheduled.
export_task_pool_size5The size of the unload task thread pool.

Storage

ParameterDefaultDescription
default_storage_mediumHDDThe default storage media that is used for a table or partition at the time of table or partition creation if no storage media is specified. Valid values: HDD and SSD. When you create a table or partition, the default storage media specified by this parameter is used if you do not specify a storage media type for the table or partition.
tablet_sched_balancer_strategydisk_and_tabletThe policy based on which load balancing is implemented among tablets. The alias of this parameter is tablet_balancer_strategy. Valid values: disk_and_tablet and be_load_score.
tablet_sched_storage_cooldown_second-1The latency of automatic cooling starting from the time of table creation. The alias of this parameter is storage_cooldown_second. Unit: second. The default value -1 specifies that automatic cooling is disabled. If you want to enable automatic cooling, set this parameter to a value greater than -1.
tablet_stat_update_interval_second300The time interval at which the FE retrieves tablet statistics from each BE. Unit: second.

Other FE static parameters

ParameterDefaultDescription
plugin_dirSTARROCKS_HOME_DIR/pluginsThe directory that stores plugin installation packages.
small_file_dirStarRocksFE.STARROCKS_HOME_DIR + "/small_files"The root directory of small files.
max_agent_task_threads_num4096The maximum number of threads that are allowed in the agent task thread pool.
auth_tokenEmpty stringThe token that is used for identity authentication within the StarRocks cluster to which the FE belongs. If this parameter is left unspecified, StarRocks generates a random token for the cluster at the time when the leader FE of the cluster is started for the first time.
tmp_dirStarRocksFE.STARROCKS_HOME_DIR + "/temp_dir"The directory that stores temporary files such as files generated during backup and restore procedures. After these procedures finish, the generated temporary files are deleted.
localezh_CN.UTF-8The character set that is used by the FE.
hive_meta_load_concurrency4The maximum number of concurrent threads that are supported for Hive metadata.
hive_meta_cache_refresh_interval_s7200The time interval at which the cached metadata of Hive external tables is updated. Unit: second.
hive_meta_cache_ttl_s86400The amount of time after which the cached metadata of Hive external tables expires. Unit: second.
hive_meta_store_timeout_s10The amount of time after which a connection to a Hive metastore times out. Unit: second.
es_state_sync_interval_second10The time interval at which the FE obtains Elasticsearch indexes and synchronizes the metadata of StarRocks external tables. Unit: second.
enable_auth_checkTRUESpecifies whether to enable the authentication check feature. Valid values: TRUE and FALSE. TRUE specifies to enable this feature, and FALSE specifies to disable this feature.
enable_metric_calculatorTRUESpecifies whether to enable the feature that is used to periodically collect metrics. Valid values: TRUE and FALSE. TRUE specifies to enable this feature, and FALSE specifies to disable this feature.

BE configuration items

Some BE configuration items are dynamic parameters which you can set them by commands when BE nodes are still online. The rest of them are static parameters. You can only set the static parameters of a BE node by changing them in the corresponding configuration file be.conf, and restart the BE node to allow the change to take effect.

View BE configuration items

You can view the BE configuration items using the following command:

curl http://<BE_IP>:<BE_HTTP_PORT>/varz

Configure BE dynamic parameters

You can configure a dynamic parameter of a BE node by using the curl command.

curl -XPOST http://be_host:http_port/api/update_config?<configuration_item>=<value>

BE dynamic parameters are as follows.

Configuration itemDefaultUnitDescription
tc_use_memory_min0ByteThe minimum size of the TCMalloc-reserved memory. StarRocks does not return the released memory resource to the operating system if the size of the memory resource is less than this value.
tc_free_memory_rate0%The maximum ratio of the TCMalloc-reserved memory size to the total memory size occupied by TCMalloc. StarRocks does not return the released memory resource to the operating system if the size ratio of the released memory to the total memory used by TCMalloc is less than this value. Range: [0,100].
tc_gc_period60SecondThe duration of a TCMalloc garbage collection (GC) cycle.
report_task_interval_seconds10SecondThe time interval at which to report the state of a task. A task can be creating a table, dropping a table, loading data, or changing a table schema.
report_disk_state_interval_seconds60SecondThe time interval at which to report the storage volume state, which includes the size of data within the volume.
report_tablet_interval_seconds60SecondThe time interval at which to report the most updated version of all tablets.
report_workgroup_interval_seconds5SecondThe time interval at which to report the most updated version of all workgroups.
max_download_speed_kbps50000KB/sThe maximum download speed of each HTTP request. This value affects the performance of data replica synchronization across BE nodes.
download_low_speed_limit_kbps50KB/sThe download speed lower limit of each HTTP request. An HTTP request aborts when it constantly runs with a lower speed than this value within the time span specified in the configuration item download_low_speed_time.
download_low_speed_time300SecondThe maximum time that an HTTP request can run with a download speed lower than the limit. An HTTP request aborts when it constantly runs with a lower speed than the value of download_low_speed_limit_kbps within the time span specified in this configuration item.
status_report_interval5SecondThe time interval at which a query reports its profile, which can be used for query statistics collection by FE.
scanner_thread_pool_thread_num48N/AThe number of threads which the storage engine used for concurrent storage volume scanning. All threads are managed in the thread pool.
thrift_client_retry_interval_ms100msThe time interval at which a thrift client retries.
scanner_thread_pool_queue_size102400N/AThe number of scan tasks supported by the storage engine.
scanner_row_num16384N/AThe maximum row count returned by each scan thread in a scan.
max_scan_key_num1024N/AThe maximum number of scan key segmented by each query.
max_pushdown_conditions_per_column1024N/AThe maximum number of conditions that allow pushdown in each column. If the number of conditions exceeds this limit, the predicates are not pushed down to the storage layer.
exchg_node_buffer_size_bytes10485760ByteThe maximum buffer size on the receiver end of an exchange node for each query. This configuration item is a soft limit. A backpressure is triggered when data is sent to the receiver end with an excessive speed.
memory_limitation_per_thread_for_schema_change2GBThe maximum memory size allowed for each schema change task.
update_cache_expire_sec360SecondThe expiration time of Update Cache.
file_descriptor_cache_clean_interval3600SecondThe time interval at which to clean file descriptors that have not been used for a certain period of time.
disk_stat_monitor_interval5SecondThe time interval at which to monitor health status of disks.
unused_rowset_monitor_interval30SecondThe time interval at which to clean the expired rowsets.
max_percentage_of_error_disk0%The maximum percentage of error that is tolerable in a storage volume before the corresponding BE node quits.
default_num_rows_per_column_file_block1024N/AThe maximum number of rows that can be stored in each row block.
pending_data_expire_time_sec1800SecondThe expiration time of the pending data in the storage engine.
inc_rowset_expired_sec1800SecondThe expiration time of the incoming data. This configuration item is used in incremental clone.
tablet_rowset_stale_sweep_time_sec1800SecondThe time interval at which to sweep the stale rowsets in tablets.
snapshot_expire_time_sec172800SecondThe expiration time of snapshot files.
trash_file_expire_time_sec259200SecondThe time interval at which to clean trash files.
base_compaction_check_interval_seconds60SecondThe time interval of thread polling for a Base Compaction.
min_base_compaction_num_singleton_deltas5N/AThe minimum number of segments that trigger a Base Compaction.
max_base_compaction_num_singleton_deltas100N/AThe maximum number of segments that can be compacted in each Base Compaction.
base_compaction_interval_seconds_since_last_operation86400SecondThe time interval since the last Base Compaction. This configuration item is one of the conditions that trigger a Base Compaction.
cumulative_compaction_check_interval_seconds1SecondThe time interval of thread polling for a Cumulative Compaction.
update_compaction_check_interval_seconds60SecondThe time interval at which to check the Update Compaction of the Primary Key data model.
min_compaction_failure_interval_sec120SecondThe minimum time interval that a Tablet Compaction can be scheduled since the last compaction failure.
periodic_counter_update_period_ms500msThe time interval at which to collect the Counter statistics.
load_error_log_reserve_hours48HourThe time for which data loading logs are reserved.
streaming_load_max_mb10240MBThe maximum size of a file that can be streamed into StarRocks.
streaming_load_max_batch_size_mb100MBThe maximum size of a JSON file that can be streamed into StarRocks.
memory_maintenance_sleep_time_s10SecondThe time interval at which TCMalloc GC is triggered. StarRocks executes GC periodically, and returns the released memory memory to the operating system.
write_buffer_size104857600ByteThe buffer size of MemTable in the memory. This configuration item is the threshold to trigger a flush.
tablet_stat_cache_update_interval_second300SecondThe time interval at which to update Tablet Stat Cache.
result_buffer_cancelled_interval_time300SecondThe wait time before BufferControlBlock release data.
thrift_rpc_timeout_ms5000msThe timeout for a thrift RPC.
txn_commit_rpc_timeout_ms20000msThe timeout for a transaction commit RPC.
max_consumer_num_per_group3N/AThe maximum number of consumers in a consumer group of Routine Load.
max_memory_sink_batch_count20N/AThe maximum number of Scan Cache batches.
scan_context_gc_interval_min5MinuteThe time interval at which to clean the Scan Context.
path_gc_check_step1000N/AThe maximum number of files that can be scanned continuously each time.
path_gc_check_step_interval_ms10msThe time interval between file scans.
path_scan_interval_second86400SecondThe time interval at which GC cleans expired data.
storage_flood_stage_usage_percent95%The storage usage threshold (in percentage) that can trigger the rejection of a Load or Restore job if it is reached.
storage_flood_stage_left_capacity_bytes1073741824ByteThe minimum left capacity of the storage before the rejection of a Load or Restore job is triggered.
tablet_meta_checkpoint_min_new_rowsets_num10N/AThe minimum number of rowsets to create since the last TabletMeta Checkpoint.
tablet_meta_checkpoint_min_interval_secs600SecondThe time interval of thread polling for a TabletMeta Checkpoint.
max_runnings_transactions_per_txn_map100N/AThe maximum number of transactions that can run concurrently in each partition.
tablet_max_pending_versions1000N/AThe maximum number of pending versions that are tolerable in a Primary Key table. Pending versions refer to versions that are committed but not applied yet.
max_hdfs_file_handle1000N/AThe maximum number of HDFS file descriptors that can be opened.
parquet_buffer_stream_reserve_size1048576ByteThe size of buffer that Parquet reader reserves for each column while reading data.
be_exit_after_disk_write_hang_second60secondThe length of time that the BE waits to exit after the disk hangs.
min_cumulative_compaction_failure_interval_sec30secondThe minimum time interval at which Cumulative Compaction retries upon failures.
size_tiered_level_num7N/AThe number of levels for the Size-tiered Compaction strategy. At most one rowset is reserved for each level. Therefore, under a stable condition, there are, at most, as many rowsets as the level number specified in this configuration item.
size_tiered_level_multiple5N/AThe multiple of data size between two contiguous levels in the Size-tiered Compaction strategy.
size_tiered_min_level_size131072ByteThe data size of the minimum level in the Size-tiered Compaction strategy. Rowsets smaller than this value immediately trigger the data compaction.
storage_page_cache_limit20%N/AThe PageCache size. STRING. It can be specified as size, for example, 20G, 20480M, 20971520K, or 21474836480B. It can also be specified as the ratio (percentage) to the memory size, for example, 20%. It takes effect only when disable_storage_page_cache is set to false.

Configure BE static parameters

You can only set the static parameters of a BE by changing them in the corresponding configuration file be.conf, and restart the BE to allow the changes to take effect.

BE static parameters are as follows.

Configuration itemDefaultUnitDescription
be_port9060N/AThe BE thrift server port, which is used to receive requests from FEs.
brpc_port8060N/AThe BE BRPC port, which is used to view the network statistics of BRPCs.
brpc_num_threads-1N/AThe number of bthreads of a BRPC. The value -1 indicates the same number with the CPU threads.
priority_networksEmpty stringN/AThe CIDR-formatted IP address that is used to specify the priority IP address of a BE node if the machine that hosts the BE node has multiple IP addresses.
heartbeat_service_port9050N/AThe BE heartbeat service port, which is used to receive heartbeats from FEs.
heartbeat_service_thread_count1N/AThe thread count of the BE heartbeat service.
create_tablet_worker_count3N/AThe number of threads used to create a tablet.
drop_tablet_worker_count3N/AThe number of threads used to drop a tablet.
push_worker_count_normal_priority3N/AThe number of threads used to handle a load task with NORMAL priority.
push_worker_count_high_priority3N/AThe number of threads used to handle a load task with HIGH priority.
transaction_publish_version_worker_count8N/AThe number of threads used to publish a version.
clear_transaction_task_worker_count1N/AThe number of threads used for clearing transaction.
alter_tablet_worker_count3N/AThe number of threads used for schema change.
clone_worker_count3N/AThe number of threads used for clone.
storage_medium_migrate_count1N/AThe number of threads used for storage medium migration (from SATA to SSD).
check_consistency_worker_count1N/AThe number of threads used for check the consistency of tablets.
sys_log_dir${STARROCKS_HOME}/logN/AThe directory that stores system logs (including INFO, WARNING, ERROR, and FATAL).
user_function_dir${STARROCKS_HOME}/lib/udfN/AThe directory used to store the User-defined Functions (UDFs).
small_file_dir${STARROCKS_HOME}/lib/small_fileN/AThe directory used to store the files downloaded by the file manager.
sys_log_levelINFON/AThe severity levels into which system log entries are classified. Valid values: INFO, WARN, ERROR, and FATAL.
sys_log_roll_modeSIZE-MB-1024N/AThe mode how system logs are segmented into log rolls. Valid values include TIME-DAY, TIME-HOUR, and SIZE-MB-<size>. The default value indicates that logs are segmented into rolls which are 1 GB each.
sys_log_roll_num10N/AThe number of log rolls to reserve.
sys_log_verbose_modulesEmpty stringN/AThe module of the logs to be printed. For example, if you set this configuration item to OLAP, StarRocks only prints the logs of the OLAP module. Valid values are namespaces in BE, including starrocks, starrocks::vectorized, and pipeline.
sys_log_verbose_level10N/AThe level of the logs to be printed. This configuration item is used to control the output of logs initiated with VLOG in codes.
log_buffer_levelEmpty stringN/AThe strategy how logs are flushed. The default value indicates that logs are buffered in memory. Valid values are -1 and 0. -1 indicates that logs are not buffering in memory.
num_threads_per_core3N/AThe number threads started in each CPU core.
compress_rowbatchesTRUEN/AThe boolean value to control if to compress the row batches in RPCs between BEs. This configuration item is used for the data transmission between query layers. The value true indicates to compress the row batches. The value false indicates not to compress the row batches.
serialize_batchFALSEN/AThe boolean value to control if to serialize the row batches in RPCs between BEs. This configuration item is used for the data transmission between query layers. The value true indicates to serialize the row batches. The value false indicates not to serialize the row batches.
storage_root_path${STARROCKS_HOME}/storageN/AThe directory and medium of the storage volume. Multiple volumes are separated by semicolon (;). If the storage medium is SSD, add ,medium:ssd at the end of the directory. If the storage medium is HDD, add ,medium:hdd at the end of the directory. Example: /data1,medium:hdd;/data2,medium:ssd.
max_tablet_num_per_shard1024N/AThe maximum number of tablets in each shard. This configuration item is used to restrict the number of tablet child directories under each storage directory.
max_garbage_sweep_interval3600SecondThe maximum time interval for garbage collection on storage volumes.
min_garbage_sweep_interval180SecondThe minimum time interval for garbage collection on storage volumes.
row_nums_checkTRUEN/AThe boolean value to control if to check the row counts before and after the compaction. The value true indicates to enable the row count check. The value false indicates disable the row count check.
file_descriptor_cache_capacity16384N/AThe number of file descriptors that can be cached.
min_file_descriptor_number60000N/AThe minimum number of file descriptors in the BE process.
index_stream_cache_capacity10737418240ByteThe cache capacity for the statistical information of BloomFilter, Min, and Max.
disable_storage_page_cacheTRUEN/AThe boolean value to control if to disable PageCache. When PageCache is enabled, StarRocks caches the query results. PageCache can significantly improve the query performance when similar queries are repeated frequently. true indicates to disable PageCache.
base_compaction_num_threads_per_disk1N/AThe number of threads used for Base Compaction on each storage volume.
base_cumulative_delta_ratio0.3N/AThe ratio of cumulative file size to base file size. The ratio reaching this value is one of the conditions that trigger the Base Compaction.
max_compaction_concurrency-1N/AThe maximum concurrency of compactions (both Base Compaction and Cumulative Compaction). The value -1 indicates that no limit is imposed on the concurrency.
compaction_trace_threshold60SecondThe time threshold for each compaction. If a compaction takes more time than the time threshold, StarRocks prints the corresponding trace.
webserver_port8040N/AThe HTTP server port.
webserver_num_workers48N/AThe number of threads used by the HTTP server.
load_data_reserve_hours4HourThe reservation time for the files produced by small-scale loadings.
number_tablet_writer_threads16N/AThe number of threads used for Stream Load.
streaming_load_rpc_max_alive_time_sec1200SecondThe RPC timeout for Stream Load.
fragment_pool_thread_num_min64N/AThe minimum number of threads used for query.
fragment_pool_thread_num_max4096N/AThe maximum number of threads used for query.
fragment_pool_queue_size2048N/AThe upper limit of query number that can be processed on each BE node.
enable_partitioned_aggregationTRUEN/AThe boolean value to control if to enable the Partition Aggregation. The value true indicates to enable the Partition Aggregation. The value false indicates to disable the Partition Aggregation.
enable_token_checkTRUEN/AThe boolean value to control if to enable the token check. The value true indicates to enable the token check. The value false indicates to disable the token check.
enable_prefetchTRUEN/AThe boolean value to control if to enable the pre-fetch of the query. The value true indicates to enable the pre-fetch. The value false indicates to disable the pre-fetch.
load_process_max_memory_limit_bytes107374182400ByteThe maximum size limit of memory resources can be taken up by all load process on a BE node.
load_process_max_memory_limit_percent30%The maximum percentage limit of memory resources can be taken up by all load process on a BE node.
sync_tablet_metaFALSEN/AThe boolean value to control if to enable the synchronization of the tablet metadata. The value true indicates to enable the synchronization. The value false indicates to disable the synchronization.
routine_load_thread_pool_size10N/AThe thread pool size of Routine Load.
brpc_max_body_size2147483648ByteThe maximum body size of a BRPC.
tablet_map_shard_size32N/AThe tablet map shard size. The value must be the power of two.
enable_bitmap_union_disk_format_with_setFALSEN/AThe boolean value to control if to enable the new storage format of the BITMAP type, which can improve the performance of bitmap_union. The value true indicates to enable the new storage format. The value false indicates to disable the new storage format.
mem_limit90%N/ABE process memory upper limit. You can set it as a percentage ("80%") or a physical limit ("100GB").
flush_thread_num_per_store2N/ANumber of threads that are used for flushing MemTable in each store.
block_cache_enablefalseN/AWhether to enable Local Cache.
  • true: Local Cache is enabled.
  • false: Local Cache is disabled.
To enable Local Cache, set the value of this parameter to true.
block_cache_disk_pathN/AN/AThe paths of disks. We recommend that the number of paths you configured for this parameter is the same as the number of disks of your BE machine. Multiple paths need to be separated with semicolons (;). After you add this parameter, StarRocks automatically creates a file named cachelib_data to cache blocks.
block_cache_meta_pathN/AN/AThe storage path of block metadata. You can customize the storage path. We recommend that you store the metadata under the $STARROCKS_HOME path.
block_cache_mem_size2147483648BytesThe maximum amount of data that can be cached in the memory. Unit: bytes. The default value is 2147483648, which is 2 GB. We recommend that you set the value of this parameter to at least 20 GB. If StarRocks reads a large amount of data from disks after Local Cache is enabled, consider increasing the value.
block_cache_disk_size0BytesThe maximum amount of data that can be cached in a single disk. For example, if you configure two disk paths for the block_cache_disk_path parameter and set the value of the block_cache_disk_size parameter as 21474836480 (20 GB), a maximum of 40 GB data can be cached in these two disks. The default value is 0, which indicates that only the memory is used to cache data. Unit: bytes.
jdbc_connection_pool_size8The JDBC connection pool size. On each BE node, queries which access the external table with the same jdbc_url share the same connection pool.
jdbc_minimum_idle_connections1The minimum number of idle connections in the JDBC connection pool.
jdbc_connection_idle_timeout_ms600000The length of time after which an idle connection in the JDBC connection pool expires. If the connection idle time in the JDBC connection pool exceeds this value, the connection pool closes idle connections of more than the number specified in the configuration item jdbc_minimum_idle_connections.
query_cache_capacity536870912The size of the query cache in the BE. Unit: bytes. The default size is 512 MB. The size cannot be less than 4 MB. If the memory capacity of the BE is insufficient to provision your expected query cache size, you can increase the memory capacity of the BE.
enable_event_based_compaction_frameworkTRUEWhether to enable Event-based Compaction Framework.
  • true: Event-based Compaction Framework is enabled.
  • false: Event-based Compaction Framework is disabled.
Enabling Event-based Compaction Framework can greatly reduce the overhead of compaction in scenarios where there are many tablets or a single tablet has a large amount of data.
enable_size_tiered_compaction_strategyTRUEWhether to enable the Size-tiered Compaction strategy.
  • true: The size-tiered Compaction strategy is enabled.
  • false: The size-tiered Compaction strategy is disabled.

Set system configurations

Linux Kernel

Linux kernel 3.10 or later is recommended.

CPU configurations

Configuration itemDescriptionRecommended valueHow to set
scaling_governorThe parameter scaling_governor is used to control the CPU power mode. The default value is on-demand. The performance mode consumes more energy, produces better performance, and thereby is recommended in the deployment of StarRocks.performanceecho 'performance' | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

Memory configurations

Configuration itemDescriptionRecommended valueHow to set
overcommit_memoryMemory Overcommit allows the operating system to overcommit memory resource to processes. We recommend you enable Memory Overcommit.1echo 1 | sudo tee /proc/sys/vm/overcommit_memory
transparent_hugepageTransparent Huge Pages is enabled by default. We recommend you disable this feature because it can interfere the memory allocator, and thereby lead to a drop in performance.madviseecho 'madvise' | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
swappinessWe recommend you disable the swappiness to eliminate its affects on the performance.0echo 0 | sudo tee /proc/sys/vm/swappiness

Storage configurations

We recommend you set different scheduler algorithms in accordance with the medium of your storage volumes.

Configuration itemDescriptionRecommended valueHow to set
schedulermq-deadline scheduler algorithm suits SATA disks.mq-deadlineecho mq-deadline | sudo tee /sys/block/vdb/queue/scheduler
schedulerkyber scheduler algorithm suits NVMe or SSD disks.kyberecho kyber | sudo tee /sys/block/vdb/queue/scheduler
schedulerIf your system does not support kyber scheduler algorithm, we recommend you use none scheduler algorithm.noneecho none | sudo tee /sys/block/vdb/queue/scheduler

Network configurations

We recommend you use 10GB network in your StarRocks cluster. Otherwise, StarRocks will fail to achieve the expected performance. You can use iPerf to check the bandwidth of your cluster.

File system configurations

We recommend you use the ext4 journaling file system. You can run the following command to check the mount type:

df -Th

High concurrency configurations

If your StarRocks cluster has a high load concurrency, we recommend you set the following configurations.

echo 120000 > /proc/sys/kernel/threads-max
echo 60000 > /proc/sys/vm/max_map_count
echo 200000 > /proc/sys/kernel/pid_max

User process configuration

You can set the maximum number of user processes by running the following command:

ulimit -u 40960

File descriptor configuration

Run the following command to the maximum number of file descriptors to 65535.

ulimit -n 65535

If this configuration becomes invalid after you re-connect to the cluster, you can set the UsePAM configuration item under /etc/ssh/sshd_config to yes, and restart the SSHD service.

Others

Configuration itemRecommended valueHow to set
tcp abort on overflow1echo 1 | sudo tee /proc/sys/net/ipv4/tcp_abort_on_overflow
somaxconn1024echo 1024 | sudo tee /proc/sys/net/core/somaxconn
Edit
Feedback
In this article