> For the complete documentation index, see [llms.txt](https://docs.starrocks.io/llms.txt). This page is also available as Markdown at its `.md` URL.

# Data Cache

From v3.1.7 and v3.2.3 onwards, StarRocks introduced Data Cache to accelerate queries in shared-data clusters, replacing File Cache in earlier versions. Data Cache loads data from remote storage in blocks (on the order of MBs) as needed, while File Cache loads entire data files each time in the background, regardless of how many data rows are actually needed.

Compared to File Cache, Data Cache has the following advantages:

* Fewer reads from object storage, meaning less cost on access to object storage (if your object storage charges based on access frequency).
* Less write pressure on local disks and CPU usage, thus less impact on other loading or query tasks (because background loading threads are no longer needed).
* Optimized cache effectiveness (because File Cache may load less frequently used data in the file).
* Optimized control over the cached data, thus avoiding overwhelmed local disks caused by excessive data failed to be evicted by File Cache.

## Enable Data Cache[​](#enable-data-cache "Direct link to Enable Data Cache")

From v3.4.0 onwards, StarRocks uses a unified Data Cache instance for queries against external catalogs and cloud-native tables (in shared-data clusters).

## Configure Data Cache[​](#configure-data-cache "Direct link to Configure Data Cache")

You can configure Data Cache using the following CN(BE) configuration items:

### Cache directory[​](#cache-directory "Direct link to Cache directory")

* [storage_root_path](https://docs.starrocks.io/docs/administration/configuration/BE_parameters.md#storage_root_path) (In shared-data clusters, this item is used to specify the root path where the cached data is stored.)

### Cache disk size[​](#cache-disk-size "Direct link to Cache disk size")

* [datacache_disk_size](https://docs.starrocks.io/docs/administration/configuration/BE_parameters.md#datacache_disk_size)

## View Data Cache status[​](#view-data-cache-status "Direct link to View Data Cache status")

* Execute the following statement to view the root path that stores the cached data:

  ```sql
  SELECT * FROM information_schema.be_configs 
  WHERE NAME LIKE "%storage_root_path%";

  ```

  Usually, the cached data is stored under the sub-path `datacache/` of your `storage_root_path`.

* Execute the following statement to view the disk usage limit of Data Cache via the `DataCacheMetrics` field:

  ```sql
  SHOW BACKENDS;
  SHOW COMPUTE NODES;

  ```

## Check whether a query hits Data Cache[​](#check-whether-a-query-hits-data-cache "Direct link to Check whether a query hits Data Cache")

You can check whether a query hits Data Cache by analyzing the following metrics in the query profile:

* `CompressedBytesReadRemote`: the size of data that the system reads from the remote storage system.
* `IOTimeRemote`: the I/O time that the system spent on reading data from the remote storage system.

If these values are not zero, it implies that the query has missed the data cache and the system has to read data from the remote storage system.

## Monitor Data Cache[​](#monitor-data-cache "Direct link to Monitor Data Cache")

StarRocks provides various metrics that monitor Data Cache.

### Dashboard templates[​](#dashboard-templates "Direct link to Dashboard templates")

You can download the following Grafana Dashboard templates based on your StarRocks environment:

* [Dashboard template for StarRocks shared-data cluster on virtual machines](http://starrocks-thirdparty.oss-cn-zhangjiakou.aliyuncs.com/StarRocks-Shared_data-for-vm.json)
* [Dashboard template for StarRocks shared-data cluster on Kubernetes](http://starrocks-thirdparty.oss-cn-zhangjiakou.aliyuncs.com/StarRocks-Shared_data-for-k8s.json)

### Important metrics[​](#important-metrics "Direct link to Important metrics")

#### fslib read io_latency[​](#fslib-read-io_latency "Direct link to fslib read io_latency")

Records the read latency of Data Cache.

#### fslib write io_latency[​](#fslib-write-io_latency "Direct link to fslib write io_latency")

Records the write latency of Data Cache.

#### fslib star cache meta memory size[​](#fslib-star-cache-meta-memory-size "Direct link to fslib star cache meta memory size")

Records the estimated memory usage of Data Cache.

#### fslib star cache data disk size[​](#fslib-star-cache-data-disk-size "Direct link to fslib star cache data disk size")

Records the actual disk usage of Data Cache.

## Disable Data Cache[​](#disable-data-cache "Direct link to Disable Data Cache")

To disable Data Cache, you need to add the following configuration to the CN configuration file **cn.conf**, and restart the CN nodes:

```properties
datacache_enable = false
storage_root_path =

```

## Clear cached data[​](#clear-cached-data "Direct link to Clear cached data")

You can clear the cached data in case of emergencies. This will not affect the original data in your remote storage.

Follow these steps to clear the cached data on a CN node:

1. Remove the sub-directory that stores the data.

   Example:

   ```bash
   # Suppose `storage_root_path = /data/disk1;/data/disk2`
   rm -rf /data/disk1/datacache/
   rm -rf /data/disk2/datacache/

   ```

2. Restart the CN node.

## Usage notes[​](#usage-notes "Direct link to Usage notes")

* If the `datacache.enable` property is set to `false` for a cloud-native table, Data Cache will not be enabled for the table.
* If the `datacache.partition_duration` property is set to a specific time range, data beyond the time range will not be cached.

## Known issues[​](#known-issues "Direct link to Known issues")

### High memory usage[​](#high-memory-usage "Direct link to High memory usage")

* Description: While the cluster is running under low-load conditions, the total memory usage of the CN node is far beyond the sum of each module's memory usage.

* Identification: If the sum of `fslib star cache meta memory size` and `fslib star cache data memory size` takes a significant proportion of the total memory usage of the CN node, it might indicate this issue.

* Affected versions: v3.1.8 and earlier patch versions, and v3.2.3 and earlier patch versions

* Fixed versions: v3.1.9, v3.2.4

* Solutions:

  * Upgrade the cluster to the fixed versions.
  * If you do not want to upgrade the cluster, you can clear the directory `${storage_root_path}/starlet_cache/star_cache/meta` of the CN nodes, and restart the nodes.

## Q\&A[​](#qa "Direct link to Q\&A")

### Q1: Why does the Data Cache directory occupy much larger storage space (observed through `du` and `ls` commands) than the actual size of cached data?[​](#q1-why-does-the-data-cache-directory-occupy-much-larger-storage-space-observed-through-du-and-ls-commands-than-the-actual-size-of-cached-data "Direct link to q1-why-does-the-data-cache-directory-occupy-much-larger-storage-space-observed-through-du-and-ls-commands-than-the-actual-size-of-cached-data")

The disk space occupied by Data Cache represents the historical peak usage and is irrelevant to the current actual cached data size. For example, if 100 GB of data is cached, the data size will become 200 GB after compaction. Then, after garbage collection (GC), the data size was reduced to 100 GB. However, the disk space occupied by Data Cache will remain at its peak of 200 GB, even though the actual cached data within is 100 GB.

### Q2: Does Data Cache automatically evict data?[​](#q2-does-data-cache-automatically-evict-data "Direct link to Q2: Does Data Cache automatically evict data?")

No. Data Cache evicts data only when it reaches the disk usage limit (`80%` of the disk space by default). The eviction process does not delete the data; it merely marks the disk space that stores the old cache as empty. The new cache will then overwrite the old cache. Therefore, even if eviction occurs, the disk usage will not decrease and will not affect actual usage.

### Q3: Why does the disk usage remain at the configured maximum level and not decrease?[​](#q3-why-does-the-disk-usage-remain-at-the-configured-maximum-level-and-not-decrease "Direct link to Q3: Why does the disk usage remain at the configured maximum level and not decrease?")

Refer to Q2. Data Cache eviction mechanism does not delete the cached data but marks the old data as overwritable. Hence, the disk usage will not decrease.

### Q4: Why does the disk usage of Data Cache remain at the same level after I dropped a table and the table files are deleted from remote storage?[​](#q4-why-does-the-disk-usage-of-data-cache-remain-at-the-same-level-after-i-dropped-a-table-and-the-table-files-are-deleted-from-remote-storage "Direct link to Q4: Why does the disk usage of Data Cache remain at the same level after I dropped a table and the table files are deleted from remote storage?")

Dropping a table does not trigger the deletion of data in the Data Cache. The cache of the deleted table will gradually be evicted over time based on Data Cache's LRU (Least Recently Used) logic, and this does not affect actual usage.

### Q5: Why does the actual disk usage reach 90% or more even though 80% disk capacity is configured?[​](#q5-why-does-the-actual-disk-usage-reach-90-or-more-even-though-80-disk-capacity-is-configured "Direct link to Q5: Why does the actual disk usage reach 90% or more even though 80% disk capacity is configured?")

The disk usage by Data Cache is accurate and will not exceed the configured limit. The excessive disk usage may be caused by:

* Log files generated during runtime.
* Core files generated by CN crashes.
* Persistent indexes of Primary Key tables (stored under `${storage_root_path}/persist/`).
* Mixed deployments of BE/CN/FE instances sharing the same disk.
* Disk issues, for example, the ext3 file system is used.

You can execute `du -h . -d 1` in the disk root directory or sub-directories to check the specific space-occupying directories, and then delete the unexpected portions. You can also reduce the disk capacity limit of Data Cache by configuring `starlet_star_cache_disk_size_percent`.

### Q6: Why is there a significant difference in disk usage of Data Cache between nodes?[​](#q6-why-is-there-a-significant-difference-in-disk-usage-of-data-cache-between-nodes "Direct link to Q6: Why is there a significant difference in disk usage of Data Cache between nodes?")

It is impossible to ensure consistent cache usage across nodes due to the inherent limitations of single-node caching. As long as it does not affect query latency, differences in cache usage are acceptable. This discrepancy may be caused by:

* Different time points at which the nodes were added to the cluster.
* Differences in the number of tablets owned by different nodes.
* Differences in the size of data owned by different tablets.
* Differences in compaction and GC situations across nodes.
* Nodes experiencing crashes or Out-Of-Memory (OOM) issues.

In summary, the differences in cache usage are influenced by multiple factors.