- StarRocks
- Introduction to StarRocks
- Quick Start
- Deployment
- Deployment overview
- Prepare
- Deploy
- Deploy shared-nothing StarRocks
- Deploy and use shared-data StarRocks
- Manage
- Table Design
- Understand StarRocks table design
- Table types
- Data distribution
- Data compression
- Sort keys and prefix indexes
- Data Loading
- Concepts
- Overview of data loading
- Load data from a local file system or a streaming data source using HTTP PUT
- Load data from HDFS
- Load data from cloud storage
- Load data from Apache Kafka®
- Continuously load data from Apache Kafka®
- Load data from Apache Spark™
- Load data using INSERT
- Load data using Stream Load transaction interface
- Realtime synchronization from MySQL
- Continuously load data from Apache Flink®
- Change data through loading
- Transform data at loading
- Data Unloading
- Query Data Lakes
- Query Acceleration
- Gather CBO statistics
- Synchronous materialized views
- Asynchronous materialized views
- Colocate Join
- Lateral Join
- Query Cache
- Index
- Computing the Number of Distinct Values
- Sorted streaming aggregate
- Integrations
- Administration
- Management
- Data recovery
- User Privilege and Authentication
- Performance Tuning
- Reference
- SQL Reference
- User Account Management
- Cluster Management
- ADD SQLBLACKLIST
- ADMIN CANCEL REPAIR TABLE
- ADMIN CHECK TABLET
- ADMIN REPAIR TABLE
- ADMIN SET CONFIG
- ADMIN SET REPLICA STATUS
- ADMIN SHOW CONFIG
- ADMIN SHOW REPLICA DISTRIBUTION
- ADMIN SHOW REPLICA STATUS
- ALTER RESOURCE GROUP
- ALTER STORAGE VOLUME
- ALTER SYSTEM
- CANCEL DECOMMISSION
- CREATE FILE
- CREATE RESOURCE GROUP
- CREATE STORAGE VOLUME
- DELETE SQLBLACKLIST
- DESC STORAGE VOLUME
- DROP FILE
- DROP RESOURCE GROUP
- DROP STORAGE VOLUME
- EXPLAIN
- INSTALL PLUGIN
- KILL
- SET
- SET DEFAULT STORAGE VOLUME
- SHOW BACKENDS
- SHOW BROKER
- SHOW COMPUTE NODES
- SHOW FILE
- SHOW FRONTENDS
- SHOW FULL COLUMNS
- SHOW INDEX
- SHOW PLUGINS
- SHOW PROC
- SHOW PROCESSLIST
- SHOW RESOURCE GROUP
- SHOW SQLBLACKLIST
- SHOW STORAGE VOLUMES
- SHOW TABLE STATUS
- SHOW VARIABLES
- UNINSTALL PLUGIN
- DDL
- ALTER DATABASE
- ALTER MATERIALIZED VIEW
- ALTER TABLE
- ALTER VIEW
- ALTER RESOURCE
- ANALYZE TABLE
- BACKUP
- CANCEL ALTER TABLE
- CANCEL BACKUP
- CANCEL RESTORE
- CREATE ANALYZE
- CREATE DATABASE
- CREATE EXTERNAL CATALOG
- CREATE FUNCTION
- CREATE INDEX
- CREATE MATERIALIZED VIEW
- CREATE REPOSITORY
- CREATE RESOURCE
- CREATE TABLE
- CREATE TABLE AS SELECT
- CREATE TABLE LIKE
- CREATE VIEW
- DROP ANALYZE
- DROP CATALOG
- DROP DATABASE
- DROP FUNCTION
- DROP INDEX
- DROP MATERIALIZED VIEW
- DROP REPOSITORY
- DROP RESOURCE
- DROP STATS
- DROP TABLE
- DROP VIEW
- HLL
- KILL ANALYZE
- RECOVER
- REFRESH EXTERNAL TABLE
- RESTORE
- SET CATALOG
- SHOW ANALYZE JOB
- SHOW ANALYZE STATUS
- SHOW FUNCTION
- SHOW META
- SHOW RESOURCES
- TRUNCATE TABLE
- USE
- DML
- ALTER LOAD
- ALTER ROUTINE LOAD
- BROKER LOAD
- CANCEL LOAD
- CANCEL EXPORT
- CANCEL REFRESH MATERIALIZED VIEW
- CREATE ROUTINE LOAD
- DELETE
- DROP TASK
- EXPORT
- GROUP BY
- INSERT
- PAUSE ROUTINE LOAD
- REFRESH MATERIALIZED VIEW
- RESUME ROUTINE LOAD
- SELECT
- SHOW ALTER TABLE
- SHOW ALTER MATERIALIZED VIEW
- SHOW BACKUP
- SHOW CATALOGS
- SHOW CREATE CATALOG
- SHOW CREATE DATABASE
- SHOW CREATE MATERIALIZED VIEW
- SHOW CREATE TABLE
- SHOW CREATE VIEW
- SHOW DATA
- SHOW DATABASES
- SHOW DELETE
- SHOW DYNAMIC PARTITION TABLES
- SHOW EXPORT
- SHOW LOAD
- SHOW MATERIALIZED VIEWS
- SHOW PARTITIONS
- SHOW PROPERTY
- SHOW REPOSITORIES
- SHOW RESTORE
- SHOW ROUTINE LOAD
- SHOW ROUTINE LOAD TASK
- SHOW SNAPSHOT
- SHOW TABLES
- SHOW TABLET
- SHOW TRANSACTION
- SPARK LOAD
- STOP ROUTINE LOAD
- STREAM LOAD
- SUBMIT TASK
- UPDATE
- Auxiliary Commands
- Data Types
- Keywords
- Function Reference
- Function list
- Java UDFs
- Window functions
- Lambda expression
- Aggregate Functions
- any_value
- approx_count_distinct
- array_agg
- avg
- bitmap
- bitmap_agg
- count
- corr
- covar_pop
- covar_samp
- group_concat
- grouping
- grouping_id
- hll_empty
- hll_hash
- hll_raw_agg
- hll_union
- hll_union_agg
- max
- max_by
- min
- min_by
- multi_distinct_sum
- multi_distinct_count
- percentile_approx
- percentile_cont
- percentile_disc
- retention
- stddev
- stddev_samp
- sum
- variance, variance_pop, var_pop
- var_samp
- window_funnel
- Array Functions
- all_match
- any_match
- array_agg
- array_append
- array_avg
- array_concat
- array_contains
- array_contains_all
- array_cum_sum
- array_difference
- array_distinct
- array_filter
- array_generate
- array_intersect
- array_join
- array_length
- array_map
- array_max
- array_min
- array_position
- array_remove
- array_slice
- array_sort
- array_sortby
- array_sum
- arrays_overlap
- array_to_bitmap
- cardinality
- element_at
- reverse
- unnest
- Bit Functions
- Bitmap Functions
- base64_to_bitmap
- bitmap_agg
- bitmap_and
- bitmap_andnot
- bitmap_contains
- bitmap_count
- bitmap_from_string
- bitmap_empty
- bitmap_has_any
- bitmap_hash
- bitmap_intersect
- bitmap_max
- bitmap_min
- bitmap_or
- bitmap_remove
- bitmap_subset_in_range
- bitmap_subset_limit
- bitmap_to_array
- bitmap_to_base64
- bitmap_to_string
- bitmap_union
- bitmap_union_count
- bitmap_union_int
- bitmap_xor
- intersect_count
- sub_bitmap
- to_bitmap
- JSON Functions
- Overview of JSON functions and operators
- JSON operators
- JSON constructor functions
- JSON query and processing functions
- Map Functions
- Binary Functions
- Conditional Functions
- Cryptographic Functions
- Date Functions
- add_months
- adddate
- convert_tz
- current_date
- current_time
- current_timestamp
- date
- date_add
- date_diff
- date_format
- date_slice
- date_sub, subdate
- date_trunc
- datediff
- day
- dayname
- dayofmonth
- dayofweek
- dayofyear
- days_add
- days_diff
- days_sub
- from_days
- from_unixtime
- hour
- hours_add
- hours_diff
- hours_sub
- last_day
- makedate
- microseconds_add
- microseconds_sub
- minute
- minutes_add
- minutes_diff
- minutes_sub
- month
- monthname
- months_add
- months_diff
- months_sub
- next_day
- now
- previous_day
- quarter
- second
- seconds_add
- seconds_diff
- seconds_sub
- str_to_date
- str2date
- time_slice
- time_to_sec
- timediff
- timestamp
- timestampadd
- timestampdiff
- to_date
- to_days
- unix_timestamp
- utc_timestamp
- week
- week_iso
- weekofyear
- weeks_add
- day_of_week_iso
- weeks_diff
- weeks_sub
- year
- years_add
- years_diff
- years_sub
- Geographic Functions
- Math Functions
- String Functions
- append_trailing_char_if_absent
- ascii
- char
- char_length
- character_length
- concat
- concat_ws
- ends_with
- find_in_set
- group_concat
- hex
- hex_decode_binary
- hex_decode_string
- instr
- lcase
- left
- length
- locate
- lower
- lpad
- ltrim
- money_format
- null_or_empty
- parse_url
- repeat
- replace
- reverse
- right
- rpad
- rtrim
- space
- split
- split_part
- starts_with
- strleft
- strright
- str_to_map
- substring
- trim
- ucase
- unhex
- upper
- url_decode
- url_encode
- Pattern Matching Functions
- Percentile Functions
- Scalar Functions
- Struct Functions
- Table Functions
- Utility Functions
- cast function
- hash function
- AUTO_INCREMENT
- Generated columns
- System variables
- User-defined variables
- Error code
- System limits
- AWS IAM policies
- SQL Reference
- FAQ
- Benchmark
- Ecosystem Release Notes
- Developers
- Contribute to StarRocks
- Code Style Guides
- Use the debuginfo file for debugging
- Development Environment
- Trace Tools
Deploy StarRocks with Docker
This QuickStart tutorial guides you through the procedures to deploy StarRocks on your local machine with Docker. Before getting started, you can read StarRocks Architecture for more conceptual details.
By following these steps, you can deploy a simple StarRocks cluster with one FE node and one BE node. It can help you complete the upcoming QuickStart tutorials on creating a table and loading and querying data, and thereby acquaints you with the basic operations of StarRocks.
CAUTION
Deploying StarRocks with the Docker image used in this tutorial merely applies to the situation when you need to verify a DEMO with a small dataset. It is not recommended for a massive testing or production environment. To deploy a high-availability StarRocks cluster, see Deployment overview for other options that suit your scenarios.
Prerequisites
Before deploying StarRocks in Docker, make sure the following requirements are satisfied:
Hardware
We recommend deploying StarRocks on a machine with 8 CPU cores and 16 GB of memory or more.
Software
You must have the following software installed on your machine:
- Docker Engine (17.06.0 or later)
- MySQL client (5.5 or later)
Step 1: Download the StarRocks Docker image
Download a StarRocks Docker image from StarRocks Docker Hub. You can choose a specific version based on the tag of the image.
sudo docker run -p 9030:9030 -p 8030:8030 -p 8040:8040 \
-itd registry.starrocks.io/starrocks/allin1-ubuntu
TROUBLESHOOTING
If any of the above ports on the host machine is occupied, the system prints "docker: Error response from daemon: driver failed programming external connectivity on endpoint tender_torvalds (): Bind for 0.0.0.0:xxxx failed: port is already allocated.". You can allocate available ports on the host machine by changing the ports preceding the colons (:) in the command.
You can check if the container is created and running properly by running the following command:
sudo docker ps
As shown below, if the STATUS
of your StarRocks containers is Up
, you have successfully deployed StarRocks in the Docker container.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8962368f9208 starrocks/allin1-ubuntu:branch-3.0-0afb97bbf "/bin/sh -c ./start_…" 4 minutes ago Up 4 minutes 0.0.0.0:8037->8030/tcp, :::8037->8030/tcp, 0.0.0.0:8047->8040/tcp, :::8047->8040/tcp, 0.0.0.0:9037->9030/tcp, :::9037->9030/tcp xxxxx
Step 2: Connect to StarRocks
After StarRocks is deployed properly, you can connect to it via a MySQL client.
mysql -P9030 -h127.0.0.1 -uroot --prompt="StarRocks > "
CAUTION
If you have allocated a different port for
9030
in thedocker run
command, you must replace9030
in the above command with the port you have allocated.
You can check the status of the FE node by executing the following SQL:
SHOW PROC '/frontends'\G
Example:
StarRocks > SHOW PROC '/frontends'\G
*************************** 1. row ***************************
Name: 8962368f9208_9010_1681370634632
IP: 8962368f9208
EditLogPort: 9010
HttpPort: 8030
QueryPort: 9030
RpcPort: 9020
Role: LEADER
ClusterId: 555505802
Join: true
Alive: true
ReplayedJournalId: 99
LastHeartbeat: 2023-04-13 07:28:50
IsHelper: true
ErrMsg:
StartTime: 2023-04-13 07:24:11
Version: BRANCH-3.0-0afb97bbf
1 row in set (0.02 sec)
- If the field
Alive
istrue
, this FE node is properly started and added to the cluster. - If the field
Role
isFOLLOWER
, this FE node is eligible to be elected as the Leader FE node. - If the field
Role
isLEADER
, this FE node is the Leader FE node.
You can check the status of the BE node by executing the following SQL:
SHOW PROC '/backends'\G
Example:
StarRocks > SHOW PROC '/backends'\G
*************************** 1. row ***************************
BackendId: 10004
IP: 8962368f9208
HeartbeatPort: 9050
BePort: 9060
HttpPort: 8040
BrpcPort: 8060
LastStartTime: 2023-04-13 07:24:25
LastHeartbeat: 2023-04-13 07:29:05
Alive: true
SystemDecommissioned: false
ClusterDecommissioned: false
TabletNum: 30
DataUsedCapacity: 0.000
AvailCapacity: 527.437 GB
TotalCapacity: 1.968 TB
UsedPct: 73.83 %
MaxDiskUsedPct: 73.83 %
ErrMsg:
Version: BRANCH-3.0-0afb97bbf
Status: {"lastSuccessReportTabletsTime":"2023-04-13 07:28:26"}
DataTotalCapacity: 527.437 GB
DataUsedPct: 0.00 %
CpuCores: 16
NumRunningQueries: 0
MemUsedPct: 0.02 %
CpuUsedPct: 0.1 %
1 row in set (0.00 sec)
If the field Alive
is true
, this BE node is properly started and added to the cluster.
Stop and remove the Docker container
After completing the whole QuickStart tutorial, you can stop and remove the container that hosts your StarRocks cluster with its container ID.
NOTE
You can get the
container_id
of your Docker container by runningsudo docker ps
.
Run the following command to stop the container:
# Replace <container_id> with the container ID of your StarRocks cluster.
sudo docker stop <container_id>
If you do not need the container any longer, you can remove it by running the following command:
# Replace <container_id> with the container ID of your StarRocks cluster.
sudo docker rm <container_id>
CAUTION
The removal of the container is irreversible. Make sure you have a backup of the important data in the container before removing it.
What to do next
Having deployed StarRocks, you can continue the QuickStart tutorials on creating a table and loading and querying data.
