docs/en/faq/Others.md
This topic provides answers to some general questions.
Both are variable-length data types. When you store data of the same length, VARCHAR (32) and STRING occupy the same storage space.
Yes.
To solve this problem, perform the following steps:
For example, there is a file named original, whose text is garbled. The character set of this file is ISO-8859-1. Run the following code to obtain the character set of the file.
file --mime-encoding origin.txt
origin.txt: iso-8859-1
Run the iconv command to convert the character set of this file into UTF-8.
iconv -f iso-8859-1 -t utf-8 origin.txt > origin_utf-8.txt
After the conversion, the text of this file still appears garbled. You can then regrade the character set of this file as GBK and convert the character set into UTF-8 again.
iconv -f gbk -t utf-8 origin.txt > origin_utf-8.txt
For VARCHAR(n), StarRocks defines "n" by bytes and MySQL defines "n" by characters. According to UTF-8, one Chinese character is equal to three bytes. When StarRocks and MySQL define "n" as the same number, MySQL saves three times as many characters as StarRocks.
No, only DATE, DATETIME, and INT are supported.
Execute the SHOW DATA statement to see the corresponding storage space. You can also see the data volume, the number of copies, and the number of rows.
Note: There is a time delay in data statistics.
To request a quota increase, run the following code:
ALTER DATABASE example_db SET DATA QUOTA 10T;
StarRocks 2.2 and later support updating specific fields in a table by using the Primary Key table. StarRocks 1.9 and later support updating all fields in a table by using the Primary Key table. For more information, see Primary Key table in StarRocks 2.2.
Execute the SWAP WITH statement to swap the data between two tables or two partitions. The SWAP WITH statement is more secure than the INSERT OVERWRITE statement. Before you swap the data, check the data first and then see whether the data after the swapping is consistent with the data before the swapping.
Swap two tables: For example, there is a table named table 1. If you want to replace table 1 with another one, perform the following steps:
Create a new table named table 2.
create table2 like table1;
Use Stream Load, Broker Load, or Insert Into to load data from table 1 into table 2.
Replace table 1 with table 2.
ALTER TABLE table1 SWAP WITH table2;
By doing so, the data is loaded accurately into table 1.
Swap two partitions: For example, there is a table named table 1. If you want to replace the partition data in table 1, perform the following steps:
Create a temporary partition.
ALTER TABLE table1
ADD TEMPORARY PARTITION tp1
VALUES LESS THAN("2020-02-01");
Load the partition data from table 1 into the temporary partition.
Replace the partition of table 1 with the temporary partition.
ALTER TABLE table1
REPLACE PARTITION (p1) WITH TEMPORARY PARTITION (tp1);
This error occurs due to BDBJE's bug. To solve this problem, update the BDBJE version to 1.17 or later.
msg:Broker list path exception
path=hdfs://172.31.3.136:9000/user/hive/warehouse/zltest.db/student_info/*, broker=TNetworkAddress(hostname:172.31.4.233, port:8000)
Contact the StarRocks technical support and check whether the address and port of the namenode are correct and whether you have permission to access the address and port of the namenode.
msg:get hive partition meta data failed: java.net.UnknownHostException: emr-header-1.cluster-242
Ensure that the network is connected and upload the host file to each backend (BE) in your StarRocks cluster.
The metadata of the Apache Hive is cached in the FEs. But there is a two-hours time lag for StarRocks to update the metadata. Before StarRocks finishes the update, If you insert new data or update data in the Apache Hive table, the data in HDFS scanned by the BEs and the data obtained by the FEs are different. Therefore, this error occurs.
MySQL [bdp_dim]> select * from dim_page_func_s limit 1;
ERROR 1064 (HY000): HdfsOrcScanner::do_open failed. reason = Invalid ORC postscript length
To solve this problem, perform one of the following operations:
The default authentication plugin of MySQL 8.0 is caching_sha2_password. The default authentication plugin of MySQL 5.7 is mysql_native_password. This error occurs because you use the wrong authentication plugin.
To solve this problem, perform one of the following operations:
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'yourpassword';
my.cnf file.vim my.cnf
[mysqld]
default_authentication_plugin=mysql_native_password
If you execute the DROP TABLE statement to delete a table, StarRocks takes a while to release the allocated disk space. To release the allocated disk space immediately, execute the DROP TABLE FORCE statement to delete a table. When you execute the DROP TABLE FORCE statement, the StarRocks deletes the table directly without checking whether there are unfinished events in it. We recommend that you execute the DROP TABLE FORCE statement with caution. Because once the table is deleted, you cannot restore it.
Run the select current_version(); command or the CLI command ./bin/show_fe_version.sh to view the current version.
The metadata is stored in the memory used by the FE. You can set the memory size of the FE according to the number of tablets as shown in the table below. For example, if the number of tablets is below 1 million, you should allocate a minimum of 16 GB memory to the FE. You can configure the values of the parameters -Xms and -Xmx in the JAVA_OPTS configuration item in the fe.conf file, and the values of the parameters -Xms and -Xmx should be consistent. Note that the configuration should be same across all FEs because any of the FEs can be elected as a Leader.
| Number of tablets | Memory size of each FE |
|---|---|
| below 1 million | 16 GB |
| 1 ～ 2 million | 32 GB |
| 2 ～ 5 million | 64 GB |
| 5 ～ 10 million | 128 GB |
StarRocks supports querying data by using multiple threads. Query time refers to the time used by multiple threads to query data.
No.
You can test the concurrency limitations based on the actual business scenarios or simulated business scenarios. According to the feedback of some users, maximum of 20,000 QPS or 30,000 QPS can be achieved.
The speed to read disks for the first query relates to the performance of disks. After the first query, the page cache is generated for the subsequent queries, so the query is faster than before.
StarRocks supports single node deployment, so you need to configure at least one BE. BEs need to be run with AVX2, so we recommend that you deploy BEs on machines with 8-core and 16GB or higher configurations.
You can create a new user account and then set the data permission by granting permissions on the table query to the user.
enable_profile to true?The report is only submitted to the leader FE for access.
Run the show create table xxx command.
Only StarRocks 2.1 or later version supports specifying the default value for a function. For versions earlier than StarRocks 2.1, you can only specify a constant for a function.
You can remove the directory trash using rm -rf command. If you have already restored your data from snapshot, you can remove the directory snapshot.
Yes. You can add the disks to the directory specified by the BE configuration item storage_root_path.
Currently, for tables with the expression partitioning strategy, partitions created during loading tasks conflict with those created during ALTER TABLE tasks. Since loading tasks take priority, any conflicting ALTER tasks will fail. To prevent this issue, consider the following workarounds:
CREATE TABLE t(
event_time DATETIME
)
PARTITION BY date_trunc('hour', event_time);
EXPLAIN ANALYZE
INSERT INTO t (event_time)
SELECT DATE_ADD(NOW(), INTERVAL d hour)
FROM table(generate_series(0, 8)) AS g(d);
SHOW PARTITIONS FROM t;