介紹
本期主角:
ShardingCore一款ef-core下高效能、輕量級針對分表分庫讀寫分離的解決方案，具有零依賴、零學習成本、零業務程式碼入侵WTMWalkingTec.Mvvm框架（簡稱WTM）是基於.net core的快速開發框架。支援Layui(前後端不分離), React(前後端分離),VUE(前後端分離)，內建程式碼生成器，最大程度的提高開發效率，是一款高效開發的利器。
ShardingCore最新版本針對路由有了極大效能的優化由原先的Expression改成自定義的RouteExpression去除了Compile帶來的效能損耗
我不是efcore怎麼辦
這邊肯定有小夥伴要問有沒有不是efcore的,我這邊很確信的和你講有並且適應所有的ADO.NET包括sqlhelper
ShardingConnector 一款基於ado.net下的高效能分表分庫解決方案目前已有demo案例，這個框架你可以認為是.Net版本的ShardingSphere但是目前僅實現了ShardingSphere-JDBC,後續我將會實現ShardingSphere-Proxy希望各位.Neter多多關注
背景
之前我不是發了一篇部落格嗎.Net分表分庫動態化處理 下面有個小夥伴留言，希望可以讓我支援一下WTM 框架。我心想著處於對自己的框架的自信,並且之前有過對abpvnex和furion等一系列框架的相容適配的嘗試，原則上將只要你是efcore那麼基本上都可以支援，所以秉著嘗試以下的態度這邊就上手了，先說下結論就是可以支援，完美不完美不清楚因為本人這個框架用的不多不知道是否是完美適配。
原理
ShardingCore
ShardingCore的整體架構是一個殼dbcontext帶多個dbcontext，殼dbcontext不進行增刪改查，由內部的dbcontext自己去執行，這個因為efcore的一個物件對應一個表所限制的。我們這邊把殼dbcontext稱作shellDbContext，執行的dbcontext叫做executorDbContext,對於ShardingCore還有一個要求就是需要初始化啟動的時候Start()，Start()內部需要IServiceProvider來獲取DbContext，所以說整個框架離不開ioc，那麼就需要啟動的時候依賴注入DbContext,又因為依賴注入如果是預設的只能允許單個建構函式。這就是ShardingCore在相容使用的時候需要注意的地方。
WTM
WTM這邊我不是很熟悉，花了大概半個小時到一個小時左右的時間，進行了程式碼的翻閱，大概瞭解了其中的實現，DbContext的建立由獨立的建構函式來實現，預設通過DbContext的內部方法 OnConfiguring(DbContextOptionsBuilder optionsBuilder)來進行初始化，框架裡面將DbContext抽象成了IDataContext介面，框架默IDataContext介面預設依賴注入為NullDbContext如果需要使用會自行通過反射呼叫建構函式引數為CS型別的那一個。整體的efcore上的一些處理通過除錯程式碼和原始碼的檢視基本上了解了
開始接入
建立專案
那麼我們首先通過WTM生成一個腳手架的簡單專案，這邊生成了一個mvc的專案。
新增依賴
新增ShardingCore依賴,需要x.5.0.6+版本，x代表efcore的版本
Install-Package ShardingCore -Version 6.5.0.6
新增抽象分表DbContext
這邊和AbpVNext時候繼承一樣，因為c#不支援多繼承，好在ShardingCore是介面依賴不存在實現依賴所以任何框架都可以相容。
public abstract class AbstractShardingFrameworkContext:FrameworkContext, IShardingDbContext, ISupportShardingReadWrite
{
protected IShardingDbContextExecutor ShardingDbContextExecutor
{
get;
}
public AbstractShardingFrameworkContext(CS cs)
: base(cs)
{
ShardingDbContextExecutor =
(IShardingDbContextExecutor)Activator.CreateInstance(
typeof(ShardingDbContextExecutor<>).GetGenericType0(this.GetType()),this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(string cs, DBTypeEnum dbtype)
: base(cs, dbtype)
{
ShardingDbContextExecutor =
(IShardingDbContextExecutor)Activator.CreateInstance(
typeof(ShardingDbContextExecutor<>).GetGenericType0(this.GetType()),this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(string cs, DBTypeEnum dbtype, string version = null)
: base(cs, dbtype, version)
{
ShardingDbContextExecutor =
(IShardingDbContextExecutor)Activator.CreateInstance(
typeof(ShardingDbContextExecutor<>).GetGenericType0(this.GetType()),this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(DbContextOptions options) : base(options)
{
var wrapOptionsExtension = options.FindExtension<ShardingWrapOptionsExtension>();
if (wrapOptionsExtension != null)
{
ShardingDbContextExecutor =
(IShardingDbContextExecutor)Activator.CreateInstance(
typeof(ShardingDbContextExecutor<>).GetGenericType0(this.GetType()),this);
}
IsExecutor = wrapOptionsExtension == null;
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (this.CSName!=null)
{
base.OnConfiguring(optionsBuilder);
optionsBuilder.UseSharding<DataContext>();
}
}
/// <summary>
/// 讀寫分離優先順序
/// </summary>
public int ReadWriteSeparationPriority
{
get => ShardingDbContextExecutor.ReadWriteSeparationPriority;
set => ShardingDbContextExecutor.ReadWriteSeparationPriority = value;
}
/// <summary>
/// 是否使用讀寫分離
/// </summary>
public bool ReadWriteSeparation
{
get => ShardingDbContextExecutor.ReadWriteSeparation;
set => ShardingDbContextExecutor.ReadWriteSeparation = value;
}
/// <summary>
/// 是否是真正的執行者
/// </summary>
public bool IsExecutor { get;}
public DbContext GetDbContext(string dataSourceName, bool parallelQuery, IRouteTail routeTail)
{
return ShardingDbContextExecutor.CreateDbContext(parallelQuery, dataSourceName, routeTail);
}
/// <summary>
/// 根據物件建立通用的dbcontext
/// </summary>
/// <typeparam name="TEntity"></typeparam>
/// <param name="entity"></param>
/// <returns></returns>
public DbContext CreateGenericDbContext<TEntity>(TEntity entity) where TEntity : class
{
return ShardingDbContextExecutor.CreateGenericDbContext(entity);
}
public IVirtualDataSource GetVirtualDataSource()
{
return ShardingDbContextExecutor.GetVirtualDataSource();
}
public override EntityEntry Add(object entity)
{
if (IsExecutor)
base.Add(entity);
return CreateGenericDbContext(entity).Add(entity);
}
public override EntityEntry<TEntity> Add<TEntity>(TEntity entity)
{
if (IsExecutor)
return base.Add(entity);
return CreateGenericDbContext(entity).Add(entity);
}
public override ValueTask<EntityEntry<TEntity>> AddAsync<TEntity>(TEntity entity, CancellationToken cancellationToken = new CancellationToken())
{
if (IsExecutor)
return base.AddAsync(entity, cancellationToken);
return CreateGenericDbContext(entity).AddAsync(entity, cancellationToken);
}
public override ValueTask<EntityEntry> AddAsync(object entity, CancellationToken cancellationToken = new CancellationToken())
{
if (IsExecutor)
return base.AddAsync(entity, cancellationToken);
return CreateGenericDbContext(entity).AddAsync(entity, cancellationToken);
}
private Dictionary<DbContext, IEnumerable<TEntity>> AggregateToDic<TEntity>(IEnumerable<TEntity> entities) where TEntity:class
{
return entities.Select(o =>
{
var dbContext = CreateGenericDbContext(o);
return new
{
DbContext = dbContext,
Entity = o
};
}).GroupBy(g => g.DbContext).ToDictionary(o => o.Key, o => o.Select(g => g.Entity));
}
public override void AddRange(params object[] entities)
{
if (IsExecutor)
{
base.AddRange(entities);
return;
}
var aggregateToDic = AggregateToDic(entities);
foreach (var aggregateKv in aggregateToDic)
{
aggregateKv.Key.AddRange(aggregateKv.Value);
}
}
public override void AddRange(IEnumerable<object> entities)
{
if (IsExecutor)
{
base.AddRange(entities);
return;
}
var aggregateToDic = AggregateToDic(entities);
foreach (var aggregateKv in aggregateToDic)
{
aggregateKv.Key.AddRange(aggregateKv.Value);
}
}
public override async Task AddRangeAsync(params object[] entities)
{
if (IsExecutor)
{
await base.AddRangeAsync(entities);
return;
}
var aggregateToDic = AggregateToDic(entities);
foreach (var aggregateKv in aggregateToDic)
{
await aggregateKv.Key.AddRangeAsync(aggregateKv.Value);
}
}
public override async Task AddRangeAsync(IEnumerable<object> entities, CancellationToken cancellationToken = new CancellationToken())
{
if (IsExecutor)
{
await base.AddRangeAsync(entities, cancellationToken);
return;
}
var aggregateToDic = AggregateToDic(entities);
foreach (var aggregateKv in aggregateToDic)
{
await aggregateKv.Key.AddRangeAsync(aggregateKv.Value,cancellationToken);
}
}
public override EntityEntry<TEntity> Attach<TEntity>(TEntity entity)
{
if (IsExecutor)
return base.Attach(entity);
return CreateGenericDbContext(entity).Attach(entity);
}
public override EntityEntry Attach(object entity)
{
if (IsExecutor)
return base.Attach(entity);
return CreateGenericDbContext(entity).Attach(entity);
}
public override void AttachRange(params object[] entities)
{
if (IsExecutor)
{
base.AttachRange(entities);
return;
}
var aggregateToDic = AggregateToDic(entities);
foreach (var aggregateKv in aggregateToDic)
{
aggregateKv.Key.AttachRange(aggregateKv.Value);
}
}
public override void AttachRange(IEnumerable<object> entities)
{
if (IsExecutor)
{
base.AttachRange(entities);
return;
}
var aggregateToDic = AggregateToDic(entities);
foreach (var aggregateKv in aggregateToDic)
{
aggregateKv.Key.AttachRange(aggregateKv.Value);
}
}
public override EntityEntry<TEntity> Entry<TEntity>(TEntity entity)
{
if (IsExecutor)
return base.Entry(entity);
return CreateGenericDbContext(entity).Entry(entity);
}
public override EntityEntry Entry(object entity)
{
if (IsExecutor)
return base.Entry(entity);
return CreateGenericDbContext(entity).Entry(entity);
}
public override EntityEntry<TEntity> Update<TEntity>(TEntity entity)
{
if (IsExecutor)
return base.Update(entity);
return CreateGenericDbContext(entity).Update(entity);
}
public override EntityEntry Update(object entity)
{
if (IsExecutor)
return base.Update(entity);
return CreateGenericDbContext(entity).Update(entity);
}
public override void UpdateRange(params object[] entities)
{
if (IsExecutor)
{
base.UpdateRange(entities);
return;
}
var aggregateToDic = AggregateToDic(entities);
foreach (var aggregateKv in aggregateToDic)
{
aggregateKv.Key.UpdateRange(aggregateKv.Value);
}
}
public override void UpdateRange(IEnumerable<object> entities)
{
if (IsExecutor)
{
base.UpdateRange(entities);
return;
}
var aggregateToDic = AggregateToDic(entities);
foreach (var aggregateKv in aggregateToDic)
{
aggregateKv.Key.UpdateRange(aggregateKv.Value);
}
}
public override EntityEntry<TEntity> Remove<TEntity>(TEntity entity)
{
if (IsExecutor)
return base.Remove(entity);
return CreateGenericDbContext(entity).Remove(entity);
}
public override EntityEntry Remove(object entity)
{
if (IsExecutor)
return base.Remove(entity);
return CreateGenericDbContext(entity).Remove(entity);
}
public override void RemoveRange(params object[] entities)
{
if (IsExecutor)
{
base.RemoveRange(entities);
return;
}
var aggregateToDic = AggregateToDic(entities);
foreach (var aggregateKv in aggregateToDic)
{
aggregateKv.Key.RemoveRange(aggregateKv.Value);
}
}
public override void RemoveRange(IEnumerable<object> entities)
{
if (IsExecutor)
{
base.RemoveRange(entities);
return;
}
var aggregateToDic = AggregateToDic(entities);
foreach (var aggregateKv in aggregateToDic)
{
aggregateKv.Key.RemoveRange(aggregateKv.Value);
}
}
public override int SaveChanges()
{
if (IsExecutor)
return base.SaveChanges();
return this.SaveChanges(true);
}
public override int SaveChanges(bool acceptAllChangesOnSuccess)
{
if (IsExecutor)
return base.SaveChanges(acceptAllChangesOnSuccess);
//ApplyShardingConcepts();
int i = 0;
//如果是內部開的事務就內部自己消化
if (Database.AutoTransactionsEnabled&&Database.CurrentTransaction==null&&ShardingDbContextExecutor.IsMultiDbContext)
{
using (var tran = Database.BeginTransaction())
{
i = ShardingDbContextExecutor.SaveChanges(acceptAllChangesOnSuccess);
tran.Commit();
}
}
else
{
i = ShardingDbContextExecutor.SaveChanges(acceptAllChangesOnSuccess);
}
return i;
}
public override Task<int> SaveChangesAsync(CancellationToken cancellationToken = new CancellationToken())
{
if (IsExecutor)
return base.SaveChangesAsync(cancellationToken);
return this.SaveChangesAsync(true, cancellationToken);
}
public override async Task<int> SaveChangesAsync(bool acceptAllChangesOnSuccess, CancellationToken cancellationToken = new CancellationToken())
{
if (IsExecutor)
return await base.SaveChangesAsync(acceptAllChangesOnSuccess,cancellationToken);
//ApplyShardingConcepts();
int i = 0;
//如果是內部開的事務就內部自己消化
if (Database.AutoTransactionsEnabled && Database.CurrentTransaction==null && ShardingDbContextExecutor.IsMultiDbContext)
{
using (var tran = await Database.BeginTransactionAsync(cancellationToken))
{
i = await ShardingDbContextExecutor.SaveChangesAsync(acceptAllChangesOnSuccess, cancellationToken);
await tran.CommitAsync(cancellationToken);
}
}
else
{
i = await ShardingDbContextExecutor.SaveChangesAsync(acceptAllChangesOnSuccess, cancellationToken);
}
return i;
}
public override void Dispose()
{
if (IsExecutor)
{
base.Dispose();
}
else
{
ShardingDbContextExecutor.Dispose();
base.Dispose();
}
}
public override async ValueTask DisposeAsync()
{
if (IsExecutor)
{
await base.DisposeAsync();
}
else
{
await ShardingDbContextExecutor.DisposeAsync();
await base.DisposeAsync();
}
}
public Task RollbackAsync(CancellationToken cancellationToken = new CancellationToken())
{
return ShardingDbContextExecutor.RollbackAsync(cancellationToken);
}
public Task CommitAsync(CancellationToken cancellationToken = new CancellationToken())
{
return ShardingDbContextExecutor.CommitAsync(cancellationToken);
}
public void NotifyShardingTransaction()
{
ShardingDbContextExecutor.NotifyShardingTransaction();
}
public void Rollback()
{
ShardingDbContextExecutor.Rollback();
}
public void Commit()
{
ShardingDbContextExecutor.Commit();
}
}
簡單說一下這邊實現了WTM的所有建構函式,因為ShardingCore原生需要DbContextOption,當然也是可以支援實現類由自定義DbContext,建構函式中如果使用了DbContextOption那麼就是由依賴注入或者ShardingCore建立的DbContext,其餘的全部是WTM建立的，所以這邊都需要實現並且其餘的建構函式直接設定為ShellDbContext
又因為WTM預設的建立會賦值CSName所以需要對其後續進行UseSharding處理這是ShardingCore針對ShellDbContext必須要處理的
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (this.CSName!=null)
{
base.OnConfiguring(optionsBuilder);
optionsBuilder.UseSharding<DataContext>();
}
}
實現DataContext
很簡單隻需要繼承抽象類和實現IShardingTableDbContext介面即可,實現該介面才能支援分表否則僅支援分庫
public class DataContext : AbstractShardingFrameworkContext,IShardingTableDbContext
{
}
編寫自定義DbContext建立
因為WTM框架的DbContext擁有多個建構函式所以需要自定義，由ShardingCore提供
程式碼其實很簡單就是如何建立一個DbContext，因為ShardingCore預設的會校驗只能擁有一個建構函式並且建構函式只能是DbContextOptions或者DbContextOptions<>
public class WTMDbContextCreator<TShardingDbContext>:IDbContextCreator<TShardingDbContext> where TShardingDbContext : DbContext, IShardingDbContext
{
public DbContext CreateDbContext(DbContext shellDbContext, ShardingDbContextOptions shardingDbContextOptions)
{
var context = new DataContext((DbContextOptions<DataContext>)shardingDbContextOptions.DbContextOptions);
context.RouteTail = shardingDbContextOptions.RouteTail;
return context;
}
}
編寫分表測試類
public class Todo
{
public string Id { get; set; }
public string Name { get; set; }
}
然後再DbContext出簡單設定一下
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
//你用dbset也是可以的
modelBuilder.Entity<Todo>(e =>
{
e.HasKey(o => o.Id);
e.ToTable(nameof(Todo));
});
}
新增分表路由
public class TodoRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<Todo>
{
public TodoRoute() : base(2, 10)
{
}
public override void Configure(EntityMetadataTableBuilder<Todo> builder)
{
builder.ShardingProperty(o => o.Id);
}
}
StartUp
接下來就是激動人心的時候了，首先我們說過ShardingCore需要依賴注入，由因為DbContext是多建構函式
services.AddScoped<DataContext>(sp =>
{
var dbContextOptionsBuilder = new DbContextOptionsBuilder<DataContext>();
dbContextOptionsBuilder.UseMySql(
"server=127.0.0.1;port=3306;database=shardingTest;userid=root;password=root;",
new MySqlServerVersion(new Version()));
dbContextOptionsBuilder.UseSharding<DataContext>();
return new DataContext(dbContextOptionsBuilder.Options);
});
注意依賴注入獲取的是ShellDbContext所以我們需要對其進行UseSharding
再來我們需要配置ShardingCore
services.AddShardingConfigure<DataContext>()
.AddEntityConfig(o =>
{
o.CreateShardingTableOnStart = true;
o.EnsureCreatedWithOutShardingTable = true;
o.AddShardingTableRoute<TodoRoute>();
})
.AddConfig(o =>
{
o.AddDefaultDataSource("ds0",
"server=127.0.0.1;port=3306;database=shardingTest;userid=root;password=root;");
o.ConfigId = "c1";
o.UseShardingQuery((conn, build) =>
{
build.UseMySql(conn, new MySqlServerVersion(new Version())).UseLoggerFactory(efLogger);
});
o.UseShardingTransaction((conn,build)=>
build.UseMySql(conn,new MySqlServerVersion(new Version())).UseLoggerFactory(efLogger)
);
o.ReplaceTableEnsureManager(sp => new MySqlTableEnsureManager<DataContext>());
}).EnsureConfig();
這邊的配置就是ShardingCore很簡單可以查詢文件或者過往的部落格
這個時候有人要說了為什麼不使用AddShardingDbContext因為多建構函式預設不支援需要手動處理。
替換ShardingCore的DbContext建立,我們剛才寫的
services.Replace(ServiceDescriptor.Singleton<IDbContextCreator<DataContext>, WTMDbContextCreator<DataContext>>());
再然後替換WTM的IDataContext
//這是WTM的預設的需要替換掉
//services.TryAddScoped<IDataContext, NullContext>();
services.Replace(ServiceDescriptor.Scoped<IDataContext>(sp =>
{
return sp.GetService<DataContext>();
}));
然後啟動初始化ShardingCore
app.ApplicationServices.GetRequiredService<IShardingBootstrapper>().Start();
編寫測試demo
public async Task<ActionResult> Login(LoginVM vm)
{
var dataContext = Wtm.DC;
var todos = new List<Todo>();
for (int i = 0; i < 100; i++)
{
var todo = new Todo();
todo.Id = Guid.NewGuid().ToString("n");
todo.Name = todo.Id;
todos.Add(todo);
}
await dataContext.Set<Todo>().AddRangeAsync(todos);
await dataContext.SaveChangesAsync();
var listAsync = await dataContext.Set<Todo>().Take(2).ToListAsync();
....
}
啟動執行
完美建立分表並且可以插入查詢完全和使用WTM一樣
最後的最後
demo地址 https://github.com/xuejmnet/ShardingWTM
您都看到這邊了確定不點個star或者贊嗎,一款.Net不得不學的分庫分表解決方案,簡單理解為sharding-jdbc在.net中的實現並且支援更多特性和更優秀的資料聚合,擁有原生效能的97%,並且無業務侵入性,支援未分片的所有efcore原生查詢