- Contribute to StarRocks
- Code Style Guides
- Use the debuginfo file for debugging
- Development Environment
- Trace Tools
StarRocks System Architecture
System Architecture Diagram
The StarRocks cluster is composed of FE and BE, and can be accessed via MySQL client.
- Connects to the MySQL client, parses and executes SQL statements.
- Manages metadata, executes SQL DDL commands, and uses Catalog to record information about databases, tables, partitions, and tablet replications.
- High-availability deployment. Use unique replication protocol to select leader and synchronize metadata.
- FE leader node modifies metadata; FE follower node performs read operations. Metadata read and write satisfies sequential consistency.
- The number of FE nodes is 2n+1, which can tolerate n node failures. If the FE leader node fails, one of the follower nodes gets selected as the new leader node to complete the failover.
- FE's SQL layer parses, analyzes, and rewrites users’ SQL statements. It also performs semantic analysis and relational algebra optimization, and produces logical execution plans.
- FE Planner is responsible for transforming logical plans into physical plans that can be distributed to BE.
- FE supervises BE, manages BE's online and offline, and maintains the number of tablet copies based on BE's health status.
- FE coordinates data import to ensure data consistency.
- Manages tablet copies. Tablets are sub-tables with columnar storage, formed by table partitioning and hashing.
- Creates or deletes sub-tables, instructed by FE.
- Receives physical execution plans distributed by FE. The BE coordinator cooperates with other BE workers to complete executions.
- Obtains data from the local column storage engine, and quickly filters it through indexing and predicate pushdown.
- Executes compact tasks in the background to reduce read amplification during queries.
- During data import, FE designates the BE coordinator to write fanout-form data to the BE where the multiple copies of the tablet are located.
- Hdfs Broker