- StarRocks
- Introduction to StarRocks
- Quick Start
- Table Design
- Data Loading
- Concepts
- Overview of data loading
- Load data from a local file system or a streaming data source using HTTP PUT
- Load data from HDFS or cloud storage
- Continuously load data from Apache Kafka®
- Bulk load using Apache Sparkâ„¢
- Load data using INSERT
- Load data using Stream Load transaction interface
- Synchronize data from MySQL in real time
- Continuously load data from Apache Flink®
- Change data through loading
- Transform data at loading
- Data Unloading
- Query Data Sources
- Query Acceleration
- Administration
- Deployment
- Management
- Data recovery
- User Privilege and Authentication
- Performance Tuning
- Reference
- SQL Reference
- User Account Management
- Cluster Management
- ADD SQLBLACKLIST
- ADMIN CANCEL REPAIR TABLE
- ADMIN CHECK TABLET
- ADMIN REPAIR TABLE
- ADMIN SET CONFIG
- ADMIN SET REPLICA STATUS
- ADMIN SHOW CONFIG
- ADMIN SHOW REPLICA DISTRIBUTION
- ADMIN SHOW REPLICA STATUS
- ALTER RESOURCE GROUP
- ALTER SYSTEM
- CANCEL DECOMMISSION
- CREATE FILE
- CREATE RESOURCE GROUP
- DELETE SQLBLACKLIST
- DROP FILE
- DROP RESOURCE GROUP
- EXPLAIN
- INSTALL PLUGIN
- KILL
- SET
- SHOW BACKENDS
- SHOW BROKER
- SHOW COMPUTE NODES
- SHOW FILE
- SHOW FRONTENDS
- SHOW FULL COLUMNS
- SHOW INDEX
- SHOW PLUGINS
- SHOW PROC
- SHOW PROCESSLIST
- SHOW RESOURCE GROUP
- SHOW SQLBLACKLIST
- SHOW TABLE STATUS
- SHOW VARIABLES
- UNINSTALL PLUGIN
- DDL
- ALTER DATABASE
- ALTER MATERIALIZED VIEW
- ALTER TABLE
- ALTER VIEW
- ALTER RESOURCE
- ANALYZE TABLE
- BACKUP
- CANCEL ALTER TABLE
- CANCEL BACKUP
- CANCEL RESTORE
- CREATE ANALYZE
- CREATE EXTERNAL CATALOG
- CREATE DATABASE
- CREATE INDEX
- CREATE MATERIALIZED VIEW
- CREATE REPOSITORY
- CREATE RESOURCE
- CREATE TABLE AS SELECT
- CREATE TABLE LIKE
- CREATE TABLE
- CREATE VIEW
- CREATE FUNCTION
- DROP ANALYZE
- DROP STATS
- DROP CATALOG
- DROP DATABASE
- DROP INDEX
- DROP MATERIALIZED VIEW
- DROP REPOSITORY
- DROP RESOURCE
- DROP TABLE
- DROP VIEW
- DROP FUNCTION
- HLL
- KILL ANALYZE
- RECOVER
- REFRESH EXTERNAL TABLE
- RESTORE
- SHOW ANALYZE JOB
- SHOW ANALYZE STATUS
- SHOW META
- SHOW RESOURCES
- SHOW FUNCTION
- TRUNCATE TABLE
- USE
- DML
- ALTER LOAD
- ALTER ROUTINE LOAD
- BROKER LOAD
- CANCEL LOAD
- CANCEL EXPORT
- CANCEL REFRESH MATERIALIZED VIEW
- CREATE ROUTINE LOAD
- DELETE
- EXPORT
- GROUP BY
- INSERT
- PAUSE ROUTINE LOAD
- REFRESH MATERIALIZED VIEW
- RESUME ROUTINE LOAD
- SELECT
- SHOW ALTER TABLE
- SHOW ALTER MATERIALIZED VIEW
- SHOW BACKUP
- SHOW CATALOGS
- SHOW CREATE MATERIALIZED VIEW
- SHOW CREATE TABLE
- SHOW CREATE VIEW
- SHOW DATA
- SHOW DATABASES
- SHOW DELETE
- SHOW DYNAMIC PARTITION TABLES
- SHOW EXPORT
- SHOW LOAD
- SHOW MATERIALIZED VIEW
- SHOW PARTITIONS
- SHOW PROPERTY
- SHOW REPOSITORIES
- SHOW RESTORE
- SHOW ROUTINE LOAD
- SHOW ROUTINE LOAD TASK
- SHOW SNAPSHOT
- SHOW TABLES
- SHOW TABLET
- SHOW TRANSACTION
- SPARK LOAD
- STOP ROUTINE LOAD
- STREAM LOAD
- Auxiliary Commands
- Data Types
- Keywords
- Function Reference
- Java UDFs
- Window functions
- Lambda expression
- Aggregate Functions
- Array Functions
- array_agg
- array_append
- array_avg
- array_concat
- array_contains
- array_contains_all
- array_cum_sum
- array_difference
- array_distinct
- array_filter
- array_intersect
- array_join
- array_length
- array_map
- array_max
- array_min
- array_position
- array_remove
- array_slice
- array_sort
- array_sortby
- array_sum
- arrays_overlap
- array_to_bitmap
- reverse
- unnest
- Bit Functions
- Bitmap Functions
- base64_to_bitmap
- bitmap_and
- bitmap_andnot
- bitmap_contains
- bitmap_count
- bitmap_from_string
- bitmap_empty
- bitmap_has_any
- bitmap_hash
- bitmap_intersect
- bitmap_max
- bitmap_min
- bitmap_or
- bitmap_remove
- bitmap_to_array
- bitmap_to_base64
- bitmap_to_string
- bitmap_union
- bitmap_union_count
- bitmap_union_int
- bitmap_xor
- intersect_count
- sub_bitmap
- to_bitmap
- Conditional Functions
- Cryptographic Functions
- Date Functions
- add_months
- adddate
- convert_tz
- current_date
- current_time
- current_timestamp
- date
- date_add
- date_format
- date_slice
- date_sub, subdate
- date_trunc
- datediff
- day
- dayname
- dayofmonth
- dayofweek
- dayofyear
- days_add
- days_diff
- days_sub
- from_days
- from_unixtime
- hour
- hours_sub
- microseconds_sub
- minute
- minutes_add
- minutes_diff
- minutes_sub
- month
- monthname
- months_add
- months_diff
- now
- quarter
- second
- seconds_add
- seconds_diff
- seconds_sub
- str_to_date
- str2date
- time_slice
- time_to_sec
- timediff
- timestamp
- timestampadd
- timestampdiff
- to_date
- to_days
- unix_timestamp
- utc_timestamp
- week
- weekofyear
- weeks_add
- weeks_diff
- weeks_sub
- year
- years_add
- years_diff
- years_sub
- Geographic Functions
- JSON Functions
- Overview of JSON functions and operators
- JSON operators
- JSON constructor functions
- JSON query and processing functions
- Map Functions
- Math Functions
- String Functions
- Pattern Matching Functions
- Percentile Functions
- Scalar Functions
- Utility Functions
- cast function
- hash function
- System variables
- User-defined variables
- Error code
- System limits
- SQL Reference
- FAQ
- Deploy
- Data Migration
- SQL
- Query Dump
- Other FAQs
- Benchmark
- Developers
- Contribute to StarRocks
- Code Style Guides
- Use the debuginfo file for debugging
- Development Environment
- Trace Tools
- Integration
Deploy StarRocks in Docker
This topic describes how to deploy StarRocks in a Docker container.
Prerequisites
Before deploying StarRocks in Docker, make sure the following requirements are satisfied.
Hardware
You can follow these steps on relatively elementary hardware, such as a machine with 8 CPU cores and 16 GB of RAM. The CPU MUST support AVX2 instruction sets.
NOTE
You can run
cat /proc/cpuinfo | grep avx2
in your terminal to check if the CPU supports the AVX2 instruction sets.
Operating system
Your machine MUST be running on OS with Linux kernel 3.10 or later.
Software
You must have Docker and MySQL client 5.5 or later versions installed on your machine.
Create a Dockerfile
Create a Dockerfile to download and install StarRocks.
FROM centos:centos7
# Prepare the StarRocks Installer. Replace the `<specific_ver_num_of_starrocks>` below with the StarRocks version that you want to deploy, for example, `2.4.0`.
ENV StarRocks_version=<specific_ver_num_of_starrocks>
# Create the directory for deployment.
ENV StarRocks_home=/data/deploy
# Specify the download path. Replace the `<url_to_download_specific_ver_of_starrocks>` below with the download path of the specific version of StarRocks that you want to deploy.
ENV StarRocks_url=<url_to_download_specific_ver_of_starrocks>
# Install StarRocks.
RUN yum -y install wget
RUN mkdir -p $StarRocks_home
RUN wget -SO $StarRocks_home/StarRocks-${StarRocks_version}.tar.gz $StarRocks_url
RUN cd $StarRocks_home && tar zxf StarRocks-${StarRocks_version}.tar.gz
# Install Java JDK.
RUN yum -y install java-1.8.0-openjdk-devel.x86_64
RUN rpm -ql java-1.8.0-openjdk-devel.x86_64 | grep bin$
# Create directories for FE meta and BE storage in StarRocks.
RUN mkdir -p $StarRocks_home/StarRocks-${StarRocks_version}/fe/meta
RUN mkdir -p $StarRocks_home/StarRocks-${StarRocks_version}/be/storage
# Install relevant tools.
RUN yum -y install mysql net-tools telnet
# Run the setup script.
COPY run_script.sh $StarRocks_home/run_script.sh
RUN chmod +x $StarRocks_home/run_script.sh
CMD $StarRocks_home/run_script.sh
CAUTION
- Replace the
<specific_ver_num_of_starrocks>
in the Dockerfile with the StarRocks version that you want to deploy, for example,2.4.0
.- Replace the
<url_to_download_specific_ver_of_starrocks>
in the Dockerfile with the download path of the specific version of StarRocks you expect to install.
Create a script
Create a setup script run_script.sh
to configure and start StarRocks.
#!/bin/bash
# Set JAVA_HOME.
JAVA_INSTALL_DIR=/usr/lib/jvm/$(rpm -aq | grep java-1.8.0-openjdk-1.8.0)
export JAVA_HOME=$JAVA_INSTALL_DIR
# Start FE.
cd $StarRocks_home/StarRocks-$StarRocks_version/fe/bin/
./start_fe.sh --daemon
# Start BE.
cd $StarRocks_home/StarRocks-$StarRocks_version/be/bin/
./start_be.sh --daemon
# Sleep until the cluster starts.
sleep 30;
# Set the BE server IP.
IP=$(ifconfig eth0 | grep 'inet' | cut -d: -f2 | awk '{print $2}')
mysql -uroot -h${IP} -P 9030 -e "alter system add backend '${IP}:9050';"
# Loop to detect the process.
while sleep 60; do
ps aux | grep starrocks | grep -q -v grep
PROCESS_STATUS=$?
if [ ${PROCESS_STATUS} -ne 0 ]; then
echo "one of the starrocks process already exit."
exit 1;
fi
done
Build a Docker image
Build a Docker image for StarRocks.
docker build --no-cache --progress=plain -t starrocks:1.0 .
Start the Docker container
Start the Docker container. You can start it by mapping the relevant ports or with the network environment of the host machine.
- Start by mapping the ports:
docker run -p 9030:9030 -p 8030:8030 -p 8040:8040 --privileged=true -itd --name starrocks-test starrocks:1.0
- Start by using the network environment of the host machine:
docker run --network host --privileged=true -itd --name starrocks-test starrocks:1.0
Log in to StarRocks
Log in to StarRocks after the Docker container is started.
mysql -uroot -h127.0.0.1 -P 9030
Verify the deployment
Run the following SQLs to check if StarRocks is successfully deployed in the Docker container.
CREATE DATABASE TEST;
USE TEST;
CREATE TABLE `sr_on_mac` (
`c0` int(11) NULL COMMENT "",
`c1` date NULL COMMENT "",
`c2` datetime NULL COMMENT "",
`c3` varchar(65533) NULL COMMENT ""
) ENGINE=OLAP
DUPLICATE KEY(`c0`)
PARTITION BY RANGE (c1) (
START ("2022-02-01") END ("2022-02-10") EVERY (INTERVAL 1 DAY)
)
DISTRIBUTED BY HASH(`c0`) BUCKETS 1
PROPERTIES (
"replication_num" = "1",
"in_memory" = "false",
"storage_format" = "DEFAULT"
);
insert into sr_on_mac values (1, '2022-02-01', '2022-02-01 10:47:57', '111');
insert into sr_on_mac values (2, '2022-02-02', '2022-02-02 10:47:57', '222');
insert into sr_on_mac values (3, '2022-02-03', '2022-02-03 10:47:57', '333');
select * from sr_on_mac where c1 >= '2022-02-02';
If no error is returned, you have successfully deployed StarRocks in Docker.