Java配置

配置示例

数据分片

以下配置中DataSourceUtil的实现为DataSourceUtil,ModuloShardingTableAlgorithm 类需用户自定义实现,详细例子 ModuloShardingTableAlgorithm

  1. DataSource getShardingDataSource() throws SQLException {
  2. ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
  3. shardingRuleConfig.getTableRuleConfigs().add(getOrderTableRuleConfiguration());
  4. shardingRuleConfig.getTableRuleConfigs().add(getOrderItemTableRuleConfiguration());
  5. shardingRuleConfig.getBindingTableGroups().add("t_order, t_order_item");
  6. shardingRuleConfig.getBroadcastTables().add("t_config");
  7. shardingRuleConfig.setDefaultDatabaseShardingStrategyConfig(new InlineShardingStrategyConfiguration("user_id", "ds${user_id % 2}"));
  8. shardingRuleConfig.setDefaultTableShardingStrategyConfig(new StandardShardingStrategyConfiguration("order_id", new ModuloShardingTableAlgorithm()));
  9. return ShardingDataSourceFactory.createDataSource(createDataSourceMap(), shardingRuleConfig, new Properties());
  10. }
  11. private static KeyGeneratorConfiguration getKeyGeneratorConfiguration() {
  12. KeyGeneratorConfiguration result = new KeyGeneratorConfiguration("SNOWFLAKE", "order_id");
  13. return result;
  14. }
  15. TableRuleConfiguration getOrderTableRuleConfiguration() {
  16. TableRuleConfiguration result = new TableRuleConfiguration("t_order", "ds${0..1}.t_order${0..1}");
  17. result.setKeyGeneratorConfig(getKeyGeneratorConfiguration());
  18. return result;
  19. }
  20. TableRuleConfiguration getOrderItemTableRuleConfiguration() {
  21. TableRuleConfiguration result = new TableRuleConfiguration("t_order_item", "ds${0..1}.t_order_item${0..1}");
  22. return result;
  23. }
  24. Map<String, DataSource> createDataSourceMap() {
  25. Map<String, DataSource> result = new HashMap<>();
  26. result.put("ds0", DataSourceUtil.createDataSource("ds0"));
  27. result.put("ds1", DataSourceUtil.createDataSource("ds1"));
  28. return result;
  29. }

读写分离

  1. DataSource getMasterSlaveDataSource() throws SQLException {
  2. MasterSlaveRuleConfiguration masterSlaveRuleConfig = new MasterSlaveRuleConfiguration("ds_master_slave", "ds_master", Arrays.asList("ds_slave0", "ds_slave1"));
  3. return MasterSlaveDataSourceFactory.createDataSource(createDataSourceMap(), masterSlaveRuleConfig, new Properties());
  4. }
  5. Map<String, DataSource> createDataSourceMap() {
  6. Map<String, DataSource> result = new HashMap<>();
  7. result.put("ds_master", DataSourceUtil.createDataSource("ds_master"));
  8. result.put("ds_slave0", DataSourceUtil.createDataSource("ds_slave0"));
  9. result.put("ds_slave1", DataSourceUtil.createDataSource("ds_slave1"));
  10. return result;
  11. }

数据脱敏

  1. DataSource getEncryptDataSource() throws SQLException {
  2. return EncryptDataSourceFactory.createDataSource(DataSourceUtil.createDataSource("demo_ds"), getEncryptRuleConfiguration(), new Properties());
  3. }
  4. private static EncryptRuleConfiguration getEncryptRuleConfiguration() {
  5. Properties props = new Properties();
  6. props.setProperty("aes.key.value", "123456");
  7. EncryptorRuleConfiguration encryptorConfig = new EncryptorRuleConfiguration("AES", props);
  8. EncryptColumnRuleConfiguration columnConfig = new EncryptColumnRuleConfiguration("plain_pwd", "cipher_pwd", "", "aes");
  9. EncryptTableRuleConfiguration tableConfig = new EncryptTableRuleConfiguration(Collections.singletonMap("pwd", columnConfig));
  10. EncryptRuleConfiguration encryptRuleConfig = new EncryptRuleConfiguration();
  11. encryptRuleConfig.getEncryptors().put("aes", encryptorConfig);
  12. encryptRuleConfig.getTables().put("t_encrypt", tableConfig);
  13. return encryptRuleConfig;
  14. }

数据分片 + 读写分离

  1. DataSource getDataSource() throws SQLException {
  2. ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
  3. shardingRuleConfig.getTableRuleConfigs().add(getOrderTableRuleConfiguration());
  4. shardingRuleConfig.getTableRuleConfigs().add(getOrderItemTableRuleConfiguration());
  5. shardingRuleConfig.getBindingTableGroups().add("t_order, t_order_item");
  6. shardingRuleConfig.getBroadcastTables().add("t_config");
  7. shardingRuleConfig.setDefaultDatabaseShardingStrategyConfig(new StandardShardingStrategyConfiguration("user_id", new PreciseModuloShardingDatabaseAlgorithm()));
  8. shardingRuleConfig.setDefaultTableShardingStrategyConfig(new StandardShardingStrategyConfiguration("order_id", new PreciseModuloShardingTableAlgorithm()));
  9. shardingRuleConfig.setMasterSlaveRuleConfigs(getMasterSlaveRuleConfigurations());
  10. return ShardingDataSourceFactory.createDataSource(createDataSourceMap(), shardingRuleConfig, new Properties());
  11. }
  12. private static KeyGeneratorConfiguration getKeyGeneratorConfiguration() {
  13. KeyGeneratorConfiguration result = new KeyGeneratorConfiguration("SNOWFLAKE", "order_id");
  14. return result;
  15. }
  16. TableRuleConfiguration getOrderTableRuleConfiguration() {
  17. TableRuleConfiguration result = new TableRuleConfiguration("t_order", "ds_${0..1}.t_order_${[0, 1]}");
  18. result.setKeyGeneratorConfig(getKeyGeneratorConfiguration());
  19. return result;
  20. }
  21. TableRuleConfiguration getOrderItemTableRuleConfiguration() {
  22. TableRuleConfiguration result = new TableRuleConfiguration("t_order_item", "ds_${0..1}.t_order_item_${[0, 1]}");
  23. return result;
  24. }
  25. List<MasterSlaveRuleConfiguration> getMasterSlaveRuleConfigurations() {
  26. MasterSlaveRuleConfiguration masterSlaveRuleConfig1 = new MasterSlaveRuleConfiguration("ds_0", "demo_ds_master_0", Arrays.asList("demo_ds_master_0_slave_0", "demo_ds_master_0_slave_1"));
  27. MasterSlaveRuleConfiguration masterSlaveRuleConfig2 = new MasterSlaveRuleConfiguration("ds_1", "demo_ds_master_1", Arrays.asList("demo_ds_master_1_slave_0", "demo_ds_master_1_slave_1"));
  28. return Lists.newArrayList(masterSlaveRuleConfig1, masterSlaveRuleConfig2);
  29. }
  30. Map<String, DataSource> createDataSourceMap() {
  31. final Map<String, DataSource> result = new HashMap<>();
  32. result.put("demo_ds_master_0", DataSourceUtil.createDataSource("demo_ds_master_0"));
  33. result.put("demo_ds_master_0_slave_0", DataSourceUtil.createDataSource("demo_ds_master_0_slave_0"));
  34. result.put("demo_ds_master_0_slave_1", DataSourceUtil.createDataSource("demo_ds_master_0_slave_1"));
  35. result.put("demo_ds_master_1", DataSourceUtil.createDataSource("demo_ds_master_1"));
  36. result.put("demo_ds_master_1_slave_0", DataSourceUtil.createDataSource("demo_ds_master_1_slave_0"));
  37. result.put("demo_ds_master_1_slave_1", DataSourceUtil.createDataSource("demo_ds_master_1_slave_1"));
  38. return result;
  39. }

数据分片 + 数据脱敏

  1. public DataSource getDataSource() throws SQLException {
  2. ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
  3. shardingRuleConfig.getTableRuleConfigs().add(getOrderTableRuleConfiguration());
  4. shardingRuleConfig.getTableRuleConfigs().add(getOrderItemTableRuleConfiguration());
  5. shardingRuleConfig.getTableRuleConfigs().add(getOrderEncryptTableRuleConfiguration());
  6. shardingRuleConfig.getBindingTableGroups().add("t_order, t_order_item");
  7. shardingRuleConfig.setDefaultDatabaseShardingStrategyConfig(new InlineShardingStrategyConfiguration("user_id", "demo_ds_${user_id % 2}"));
  8. shardingRuleConfig.setDefaultTableShardingStrategyConfig(new StandardShardingStrategyConfiguration("order_id", new PreciseModuloShardingTableAlgorithm()));
  9. shardingRuleConfig.setEncryptRuleConfig(getEncryptRuleConfiguration());
  10. return ShardingDataSourceFactory.createDataSource(createDataSourceMap(), shardingRuleConfig, new Properties());
  11. }
  12. private static TableRuleConfiguration getOrderTableRuleConfiguration() {
  13. TableRuleConfiguration result = new TableRuleConfiguration("t_order", "demo_ds_${0..1}.t_order_${[0, 1]}");
  14. result.setKeyGeneratorConfig(getKeyGeneratorConfiguration());
  15. return result;
  16. }
  17. private static TableRuleConfiguration getOrderItemTableRuleConfiguration() {
  18. TableRuleConfiguration result = new TableRuleConfiguration("t_order_item", "demo_ds_${0..1}.t_order_item_${[0, 1]}");
  19. result.setEncryptorConfig(new EncryptorConfiguration("MD5", "status", new Properties()));
  20. return result;
  21. }
  22. private static EncryptRuleConfiguration getEncryptRuleConfiguration() {
  23. Properties props = new Properties();
  24. props.setProperty("aes.key.value", "123456");
  25. EncryptorRuleConfiguration encryptorConfig = new EncryptorRuleConfiguration("AES", props);
  26. EncryptColumnRuleConfiguration columnConfig = new EncryptColumnRuleConfiguration("plain_order", "cipher_order", "", "aes");
  27. EncryptTableRuleConfiguration tableConfig = new EncryptTableRuleConfiguration(Collections.singletonMap("order_id", columnConfig));
  28. EncryptRuleConfiguration encryptRuleConfig = new EncryptRuleConfiguration();
  29. encryptRuleConfig.getEncryptors().put("aes", encryptorConfig);
  30. encryptRuleConfig.getTables().put("t_order", tableConfig);
  31. return encryptRuleConfig;
  32. }
  33. private static Map<String, DataSource> createDataSourceMap() {
  34. Map<String, DataSource> result = new HashMap<>();
  35. result.put("demo_ds_0", DataSourceUtil.createDataSource("demo_ds_0"));
  36. result.put("demo_ds_1", DataSourceUtil.createDataSource("demo_ds_1"));
  37. return result;
  38. }
  39. private static KeyGeneratorConfiguration getKeyGeneratorConfiguration() {
  40. return new KeyGeneratorConfiguration("SNOWFLAKE", "order_id", new Properties());
  41. }

治理

  1. DataSource getDataSource() throws SQLException {
  2. // OrchestrationShardingDataSourceFactory 可替换成 OrchestrationMasterSlaveDataSourceFactory 或 OrchestrationEncryptDataSourceFactory
  3. return OrchestrationShardingDataSourceFactory.createDataSource(
  4. createDataSourceMap(), createShardingRuleConfig(), new HashMap<String, Object>(), new Properties(),
  5. new OrchestrationConfiguration("orchestration-sharding-data-source", getRegistryCenterConfiguration(), false));
  6. }
  7. private RegistryCenterConfiguration getRegistryCenterConfiguration() {
  8. RegistryCenterConfiguration regConfig = new RegistryCenterConfiguration("zookeeper");//注册中心的类型可以是Zookeeper,Etcd等
  9. regConfig.setServerLists("localhost:2181");
  10. regConfig.setNamespace("sharding-sphere-orchestration");
  11. return regConfig;
  12. }

配置项说明

数据分片

ShardingDataSourceFactory

数据分片的数据源创建工厂。

名称数据类型说明
dataSourceMapMap<String, DataSource>数据源配置
shardingRuleConfigShardingRuleConfiguration数据分片配置规则
props (?)Properties属性配置

ShardingRuleConfiguration

分片规则配置对象。

名称数据类型说明
tableRuleConfigsCollection<TableRuleConfiguration>分片规则列表
bindingTableGroups (?)Collection<String>绑定表规则列表
broadcastTables (?)Collection<String>广播表规则列表
defaultDataSourceName (?)String未配置分片规则的表将通过默认数据源定位
defaultDatabaseShardingStrategyConfig (?)ShardingStrategyConfiguration默认分库策略
defaultTableShardingStrategyConfig (?)ShardingStrategyConfiguration默认分表策略
defaultKeyGeneratorConfig (?)KeyGeneratorConfiguration默认自增列值生成器配置,缺省将使用org.apache.shardingsphere.core.keygen.generator.impl.SnowflakeKeyGenerator
masterSlaveRuleConfigs (?)Collection<MasterSlaveRuleConfiguration>读写分离规则,缺省表示不使用读写分离

TableRuleConfiguration

表分片规则配置对象。

名称数据类型说明
logicTableString逻辑表名称
actualDataNodes (?)String由数据源名 + 表名组成,以小数点分隔。多个表以逗号分隔,支持inline表达式。缺省表示使用已知数据源与逻辑表名称生成数据节点。用于广播表(即每个库中都需要一个同样的表用于关联查询,多为字典表)或只分库不分表且所有库的表结构完全一致的情况
databaseShardingStrategyConfig (?)ShardingStrategyConfiguration分库策略,缺省表示使用默认分库策略
tableShardingStrategyConfig (?)ShardingStrategyConfiguration分表策略,缺省表示使用默认分表策略
keyGeneratorConfig (?)KeyGeneratorConfiguration自增列值生成器配置,缺省表示使用默认自增主键生成器
encryptorConfiguration (?)EncryptorConfiguration加解密生成器配置

StandardShardingStrategyConfiguration

ShardingStrategyConfiguration的实现类,用于单分片键的标准分片场景。

名称数据类型说明
shardingColumnString分片列名称
preciseShardingAlgorithmPreciseShardingAlgorithm精确分片算法,用于=和IN
rangeShardingAlgorithm (?)RangeShardingAlgorithm范围分片算法,用于BETWEEN

ComplexShardingStrategyConfiguration

ShardingStrategyConfiguration的实现类,用于多分片键的复合分片场景。

名称数据类型说明
shardingColumnsString分片列名称,多个列以逗号分隔
shardingAlgorithmComplexKeysShardingAlgorithm复合分片算法

InlineShardingStrategyConfiguration

ShardingStrategyConfiguration的实现类,用于配置行表达式分片策略。

名称数据类型说明
shardingColumnString分片列名称
algorithmExpressionString分片算法行表达式,需符合groovy语法,详情请参考行表达式

HintShardingStrategyConfiguration

ShardingStrategyConfiguration的实现类,用于配置Hint方式分片策略。

名称数据类型说明
shardingAlgorithmHintShardingAlgorithmHint分片算法

NoneShardingStrategyConfiguration

ShardingStrategyConfiguration的实现类,用于配置不分片的策略。

KeyGeneratorConfiguration

名称数据类型说明
columnString自增列名称
typeString自增列值生成器类型,可自定义或选择内置类型:SNOWFLAKE/UUID/LEAF_SEGMENT/LEAF_SNOWFLAKE
propsProperties自增列值生成器的相关属性配置

PropertiesConstant

属性配置项,可以为以下自增列值生成器的属性。

SNOWFLAKE
名称数据类型说明
worker.id (?)long工作机器唯一id,默认为0
max.tolerate.time.difference.milliseconds (?)long最大容忍时钟回退时间,单位:毫秒。默认为10毫秒
max.vibration.offset (?)int最大抖动上限值,范围[0, 4096),默认为1。注:若使用此算法生成值作分片值,建议配置此属性。此算法在不同毫秒内所生成的key取模2^n (2^n一般为分库或分表数) 之后结果总为0或1。为防止上述分片问题,建议将此属性值配置为(2^n)-1
LEAF_SEGMENT
名称数据类型说明
server.listString连接注册中心服务器的列表,包括IP地址和端口号,多个地址用逗号分隔。如: host1:2181,host2:2181
leaf.keyString用于标识leaf_segment所对应表里面的最大号段id
leaf.segment.id.initial.value (?)long号段id初始值,默认为1
leaf.segment.step (?)long每次分配的号段步长,默认为10000
registry.center.digest (?)String连接注册中心的权限令牌。缺省为不需要权限验证
registry.center.type (?)String注册中心类型,默认为zookeeper
LEAF_SNOWFLAKE
名称数据类型说明
server.listString连接注册中心服务器的列表,包括IP地址和端口号,多个地址用逗号分隔。如: host1:2181,host2:2181
service.idStringleaf_snowflake位于注册中心上的服务标识
max.tolerate.time.difference.milliseconds (?)long最大允许的本机与注册中心的时间误差毫秒数,默认为10000毫秒。如果时间误差超过配置时间则作业启动时将抛异常
registry.center.digest (?)String连接注册中心的权限令牌。缺省为不需要权限验证
registry.center.type (?)String注册中心类型,默认为zookeeper
max.vibration.offset (?)int最大抖动上限,范围[0, 4096),默认为1。注:若使用此算法生成值作分片值,建议配置此属性。此算法在不同毫秒内所生成的key取模2^n (2^n一般为分库或分表数) 之后结果总为0或1。为防止上述分片问题,建议将此属性值配置为(2^n)-1

EncryptRuleConfiguration

名称数据类型说明
encryptorsMap加解密器配置列表,可自定义或选择内置类型:MD5/AES
tablesMap加密表配置列表

EncryptorRuleConfiguration

名称数据类型说明
typeString加解密器类型,可自定义或选择内置类型:MD5/AES
propertiesProperties属性配置, 注意:使用AES加密器,需要配置AES加密器的KEY属性:aes.key.value

EncryptTableRuleConfiguration

名称数据类型说明
tablesMap加密列配置列表

EncryptColumnRuleConfiguration

名称数据类型说明
plainColumnString存储明文的字段
cipherColumnString存储密文的字段
assistedQueryColumnString辅助查询字段,针对ShardingQueryAssistedEncryptor类型的加解密器进行辅助查询
encryptorString加解密器名字

PropertiesConstant

属性配置项,可以为以下属性。

名称数据类型说明
sql.show (?)boolean是否开启SQL显示,默认值: false
executor.size (?)int工作线程数量,默认值: CPU核数
max.connections.size.per.query (?)int每个物理数据库为每次查询分配的最大连接数量。默认值: 1
check.table.metadata.enabled (?)boolean是否在启动时检查分表元数据一致性,默认值: false
query.with.cipher.column (?)boolean当存在明文列时,是否使用密文列查询,默认值: true

读写分离

MasterSlaveDataSourceFactory

读写分离的数据源创建工厂。

名称数据类型说明
dataSourceMapMap<String, DataSource>数据源与其名称的映射
masterSlaveRuleConfigMasterSlaveRuleConfiguration读写分离规则
props (?)Properties属性配置

MasterSlaveRuleConfiguration

读写分离规则配置对象。

名称数据类型说明
nameString读写分离数据源名称
masterDataSourceNameString主库数据源名称
slaveDataSourceNamesCollection<String>从库数据源名称列表
loadBalanceAlgorithm (?)MasterSlaveLoadBalanceAlgorithm从库负载均衡算法

PropertiesConstant

属性配置项,可以为以下属性。

名称数据类型说明
sql.show (?)boolean是否打印SQL解析和改写日志,默认值: false
executor.size (?)int用于SQL执行的工作线程数量,为零则表示无限制。默认值: 0
max.connections.size.per.query (?)int每个物理数据库为每次查询分配的最大连接数量。默认值: 1
check.table.metadata.enabled (?)boolean是否在启动时检查分表元数据一致性,默认值: false

数据脱敏

EncryptDataSourceFactory

名称数据类型说明
dataSourceDataSource数据源,任意连接池
encryptRuleConfigEncryptRuleConfiguration数据脱敏规则
props (?)Properties属性配置

EncryptRuleConfiguration

名称数据类型说明
encryptorsMap加解密器配置列表,可自定义或选择内置类型:MD5/AES
tablesMap加密表配置列表

PropertiesConstant

属性配置项,可以为以下属性。

名称数据类型说明
sql.show (?)boolean是否开启SQL显示,默认值: false
query.with.cipher.column (?)boolean当存在明文列时,是否使用密文列查询,默认值: true

治理

OrchestrationShardingDataSourceFactory

数据分片 + 治理的数据源工厂。

名称数据类型说明
dataSourceMapMap<String, DataSource>同ShardingDataSourceFactory
shardingRuleConfigShardingRuleConfiguration同ShardingDataSourceFactory
props (?)Properties同ShardingDataSourceFactory
orchestrationConfigOrchestrationConfiguration治理规则配置

OrchestrationMasterSlaveDataSourceFactory

读写分离 + 治理的数据源工厂。

名称数据类型说明
dataSourceMapMap<String, DataSource>同MasterSlaveDataSourceFactory
masterSlaveRuleConfigMasterSlaveRuleConfiguration同MasterSlaveDataSourceFactory
props (?)Properties同ShardingDataSourceFactory
orchestrationConfigOrchestrationConfiguration治理规则配置

OrchestrationEncryptDataSourceFactory

数据脱敏 + 治理的数据源工厂。

名称数据类型说明
dataSourceDataSource同EncryptDataSourceFactory
encryptRuleConfigEncryptRuleConfiguration同EncryptDataSourceFactory
props (?)Properties同ShardingDataSourceFactory
orchestrationConfigOrchestrationConfiguration治理规则配置

OrchestrationConfiguration

治理规则配置对象。

名称数据类型说明
nameString治理实例名称
overwriteboolean本地配置是否覆盖注册中心配置,如果可覆盖,每次启动都以本地配置为准
regCenterConfigRegistryCenterConfiguration注册中心配置

RegistryCenterConfiguration

用于配置注册中心。

名称数据类型说明
serverListsString连接注册中心服务器的列表,包括IP地址和端口号,多个地址用逗号分隔。如: host1:2181,host2:2181
namespace (?)String注册中心的命名空间
digest (?)String连接注册中心的权限令牌。缺省为不需要权限验证
operationTimeoutMilliseconds (?)int操作超时的毫秒数,默认500毫秒
maxRetries (?)int连接失败后的最大重试次数,默认3次
retryIntervalMilliseconds (?)int重试间隔毫秒数,默认500毫秒
timeToLiveSeconds (?)int临时节点存活秒数,默认60秒