Edit
Feedback

Flink Connector

Issue description:

The time generated by localtimestap function is normal in Flink. But it became 8 hours late when sunk to StarRocks. Flink server and StarRocks server are located in the same timezone, namely Asia/Shanghai UTC/GMT+08:00. Flink version is 1.12. Driver: flink-connector-jdbc_2.11. Can I ask how to resolve this issue?

Solution:

Please try configure the time parameter 'server-time-zone' = 'Asia/Shanghai' in Flink sink table. You may also add &serverTimezone=Asia/Shanghai in jdbc url. An example is shown below:

CREATE TABLE sk (
    sid int,
    local_dtm TIMESTAMP,
    curr_dtm TIMESTAMP
)
WITH (
    'connector' = 'jdbc',
    'url' = 'jdbc:mysql://192.168.110.66:9030/sys_device?characterEncoding=utf-8&serverTimezone=Asia/Shanghai',
    'table-name' = 'sink',
    'driver' = 'com.mysql.jdbc.Driver',
    'username' = 'sr',
    'password' = 'sr123',
    'server-time-zone' = 'Asia/Shanghai'
);

Issue description:

failed to query wartermark offset, err: Local: Bad message format

Solution:

Kafka communication needs the hostname. Users need to configure the host name resolution /etc/hosts in StarRocks cluster nodes.

Can StarRocks export 'create table statements' in batches?

Solution:

You can use Doris Tools to export the statements.

When the query is not happening, BE memory usage and cpu usage are still 100%

Issue description:

Why are they still 100%?

Solution:

This is because of regular collection of statistics. CUP is not occupied for a long term. Memory of 10G or less, once fully taken, will not be released. BE will manage it by itself. Users can limit memory usage limit through tc_use_memory_min.

tc_use_memory_min is set to be 10737418240 by default. TCmalloc defines the minimum reserved memory. StarRocks only gives idle memory back to the operation system when the memory usage exceeds this value. Users can configure it in the BE configuration file "be.conf".

Memory requested by BE is not released back to to the operation system

This is a normal phenomenon, as large blocks of memory allocated to the database from the operating system are reserved during allocation and deferred during release in order to reuse the memory and make memory allocation more convenient. It is recommended that users validate the test environment by monitoring memory usage over a longer period of time to see if the memory can be released.

Issue description:

This package needs to be obtained through Aliyun mirror address.

Solution:

Please make sure that the mirror part of /etc/maven/settings.xml is all configured to be obtained through Aliyun mirror address.

If it is, please change it to the following:

aliyunmaven central阿里云公共仓库https：//maven.aliyun.com/repository/public

Issue description:

+----------------------+--------------------------------------------------------------+
|         Option       | Required |  Default   | Type   |       Description           |
+-------------------------------------------------------------------------------------+
|  sink.buffer-flush.  |  NO      |   300000   | String | the flushing time interval, |
|  interval-ms         |          |            |        | range: [1000ms, 3600000ms]  |
+----------------------+--------------------------------------------------------------+

If this parameter is set as 15s with checkpoint interval being equal to 5 mins, does this value still take effect?

Solution:

Whichever of the three thresholds is reached first, that one will take effect first. This is not affected by the checkpoint interval value which only works for exactly once. Interval-ms is used by at_least_once.

Edit
Feedback
In this article