大家好，我是本期的微軟 MVP 實驗室研究員——馮輝。本篇文章主要介紹如何利用Windbg分析應用程式中的記憶體問題，從託管堆到非託管堆的探索以及到記憶體的分配，接下來我們一起來探索吧。
近期有幾位朋友使用我們的Magicodes.IE反饋在匯出過程中記憶體暴漲，接下來我們通過windbg來看一下什麼原因導致的。
我們先通過address -summary來看一下當前應用記憶體佔用量。
0:000> !address -summary
--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free 581 7df8`ef0c9000 ( 125.972 TB) 98.42%
<unknown> 1678 206`ffb9e000 ( 2.027 TB) 99.99% 1.58%
Image 950 0`064fd000 ( 100.988 MB) 0.00% 0.00%
Heap 58 0`050f6000 ( 80.961 MB) 0.00% 0.00%
Stack 156 0`04380000 ( 67.500 MB) 0.00% 0.00%
Other 11 0`019ad000 ( 25.676 MB) 0.00% 0.00%
TEB 52 0`00068000 ( 416.000 kB) 0.00% 0.00%
PEB 1 0`00001000 ( 4.000 kB) 0.00% 0.00%
--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_MAPPED 282 200`038a6000 ( 2.000 TB) 98.64% 1.56%
MEM_PRIVATE 1674 7`07184000 ( 28.111 GB) 1.35% 0.02%
MEM_IMAGE 950 0`064fd000 ( 100.988 MB) 0.00% 0.00%
--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_FREE 581 7df8`ef0c9000 ( 125.972 TB) 98.42%
MEM_RESERVE 295 205`f8659000 ( 2.023 TB) 99.79% 1.58%
MEM_COMMIT 2611 1`188ce000 ( 4.384 GB) 0.21% 0.00%
--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal
PAGE_READWRITE 1595 1`0dc6c000 ( 4.215 GB) 0.20% 0.00%
PAGE_EXECUTE_READ 156 0`04d66000 ( 77.398 MB) 0.00% 0.00%
PAGE_READONLY 600 0`03851000 ( 56.316 MB) 0.00% 0.00%
PAGE_NOACCESS 99 0`021f2000 ( 33.945 MB) 0.00% 0.00%
PAGE_EXECUTE_READWRITE 19 0`0027b000 ( 2.480 MB) 0.00% 0.00%
PAGE_WRITECOPY 90 0`001a0000 ( 1.625 MB) 0.00% 0.00%
PAGE_READWRITE | PAGE_GUARD 52 0`0009e000 ( 632.000 kB) 0.00% 0.00%
--- Largest Region by Usage ----------- Base Address -------- Region Size ----------
Free 189`0413c000 7c6b`01ed4000 ( 124.418 TB)
<unknown> 7dfb`2a153000 1f9`bd2ef000 ( 1.976 TB)
Image 7ffc`883c1000 0`009ba000 ( 9.727 MB)
Heap 183`0e9a1000 0`00f01000 ( 15.004 MB)
Stack 37`62980000 0`0017b000 ( 1.480 MB)
Other 183`77707000 0`01775000 ( 23.457 MB)
TEB 37`62600000 0`00002000 ( 8.000 kB)
PEB 37`627dd000 0`00001000 ( 4.000 kB)MEM_COMMIT佔用了4.384G，接下來我們利用eeheap -gc來檢查託管堆。
0:000> !eeheap -gc
GC Allocated Heap Size: Size: 0x11ac2568 (296494440) bytes.
GC Committed Heap Size: Size: 0x120e7000 (302936064) bytes.根據這些記憶體來看，似乎問題不是這裡，大量的記憶體還是出現在非託管。我們利用Windows NT堆來看一下，其實在Windows中大多數的使用者堆分配器都在ntdll.dll中的NT堆管理器API（RtlAllocateHeap/RtlFreeHeap）上建立，比如說C中的malloc/free和new/delete，另外還有COM框架中的SysAllocString以及在Win32中的LocalAlloc、GlobalAlloc和HeapAlloc，雖然說這些分配器都會建立不同的堆來儲存它們的記憶體，但是他們最終都要呼叫ntdll.dll中的NT堆來實現。
0:000> !heap -s
************************************************************************************************************************
NT HEAP STATS BELOW
************************************************************************************************************************
NtGlobalFlag enables following debugging aids for new heaps:
stack back traces
LFH Key : 0x7cfd4cc2db4ddb4d
Termination on corruption : ENABLED
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-------------------------------------------------------------------------------------
0000018378fd0000 08000002 65128 15296 64928 1720 177 17 2 c LFH
External fragmentation 11 % (177 free blocks)
00000183775c0000 08008000 64 4 64 2 1 1 0 0
000001837aa90000 08001002 1280 108 1080 26 3 2 0 0 LFH
000001837ad20000 08001002 60 8 60 2 1 1 0 0
000001837aca0000 08041002 60 8 60 5 1 1 0 0
000001887bfd0000 08001002 60 20 60 1 2 1 0 0
000001830cf30000 08001002 3324 1364 3124 19 10 3 0 0 LFH
000001830ce30000 08001002 60 8 60 5 1 1 0 0
-------------------------------------------------------------------------------------輸出結果如上所示，NT堆內容好少....什麼原因....好吧根據maoni所說，似乎是驗證出了問題。
在Windows上面所有的user mode allocations最終都是通過VirtualAlloc來獲得記憶體，bitmaps也好，GC heap也好。
不同的是你去直接呼叫VirtualAlloc還是使用其他方式去呼叫它。如果是不託管的記憶體，GC並不管轄它，當然也不知道它們的存在。
GC沒有管轄這些記憶體，所以說還是我們編寫的程式碼有問題，我們返過來再考慮一個事情，“匯出進行時，記憶體會大量增加，匯出完成後記憶體會降低下去”。我們來看一下程式碼，如下所示，其實我們現在明白的是，在我們執行期間肯定是這些記憶體一直“持有”，並沒有被釋放掉。
app.MapGet("/excel", async content =>
{
string path = Path.Combine(Directory.GetCurrentDirectory(), "test.xlsx");
List<TestDto> list = new();
for (int i = 0; i < 400; i++)
{
list.Add(new TestDto
{
ImageUrl = "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fup.enterdesk.com%2Fedpic_source%2F53%2F0a%2Fda%2F530adad966630fce548cd408237ff200.jpg&refer=http%3A%2F%2Fup.enterdesk.com&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1641193100&t=417a589da8c9ba3103ed74c33fbd6c70"
});
}
Stopwatch stopwatch = Stopwatch.StartNew();
ExcelExporter exporter = new ExcelExporter();
await exporter.Export(path, list);
stopwatch.Stop();
await content.Response.WriteAsync(stopwatch.Elapsed.TotalSeconds.ToString());
});根據記憶體的表現和我們的理論，我們繼續利用Windbg來排查一下，現在其實我們可以發現，這些物件最終還是被GC收回了，帶著理論我們繼續構思，GC是知道哪些物件可以終結的對吧？並且它們在變成不可到達時呼叫它們的終結器，在GC中會利用finalization queue來記錄這些終結物件。所以說我們是不是可以查一下？如下所示，我們來看一下。
0:000> !finalizequeue
----------------------------------
Statistics for all finalizable objects (including all objects ready for finalization):
MT Count TotalSize Class Name
00007ffc2dc23818 1 24 System.Net.Security.SafeCredentialReference
00007ffc2dac4238 1 24 System.WeakReference
00007ffc2d6eb908 1 24 System.WeakReference`1[[Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerOptions, Microsoft.AspNetCore.Server.Kestrel.Core]]
00007ffc2d6e4120 1 24 System.WeakReference`1[[System.Runtime.Loader.AssemblyLoadContext, System.Private.CoreLib]]
00007ffc2d572b68 1 24 System.WeakReference`1[[Microsoft.Extensions.DependencyInjection.ServiceProvider, Microsoft.Extensions.DependencyInjection]]
00007ffc2d429258 1 24 System.WeakReference`1[[System.IO.FileSystemWatcher, System.IO.FileSystem.Watcher]]
00007ffc2dd15c20 1 32 Microsoft.Win32.SafeHandles.SafeBCryptAlgorithmHandle
00007ffc2d6de4d8 1 32 Internal.Cryptography.Pal.Native.SafeLocalAllocHandle
00007ffc2d68fa00 1 32 Internal.Cryptography.Pal.Native.SafeCertStoreHandle
00007ffc2d3a5cc0 1 32 System.Net.Quic.Implementations.MsQuic.Internal.SafeMsQuicRegistrationHandle
00007ffc2db390c8 1 40 Interop+WinHttp+SafeWinHttpHandle
00007ffc2d69a420 1 40 Internal.Cryptography.Pal.Native.SafeCertContextHandle
00007ffc2d5bea18 1 40 System.Diagnostics.EventLog
00007ffc2dc29a38 1 48 System.Net.Security.SafeFreeCredential_SECURITY
00007ffc2d963f80 2 48 System.WeakReference`1[[System.Text.RegularExpressions.RegexReplacement, System.Text.RegularExpressions]]
00007ffc2d7a3750 2 48 System.WeakReference`1[[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelConnection, Microsoft.AspNetCore.Server.Kestrel.Core]]
00007ffc2d685e10 1 56 System.Runtime.CompilerServices.ConditionalWeakTable`2+Container[[System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1+ThreadLocalArray[[System.Char, System.Private.CoreLib]][], System.Private.CoreLib],[System.Object, System.Private.CoreLib]]
00007ffc2d44c4d0 1 56 System.Runtime.CompilerServices.ConditionalWeakTable`2+Container[[System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1+ThreadLocalArray[[System.Byte, System.Private.CoreLib]][], System.Private.CoreLib],[System.Object, System.Private.CoreLib]]
00007ffc2d96be68 1 64 CellStore`1[[System.Uri, System.Private.Uri]]
00007ffc2d96b780 1 64 FlagCellStore
00007ffc2d96af48 1 64 CellStore`1[[System.Object, System.Private.CoreLib]]
00007ffc2d96a5b8 1 64 CellStore`1[[OfficeOpenXml.ExcelCoreValue, Magicodes.IE.EPPlus]]
00007ffc2d6ddab8 2 64 Internal.Cryptography.Pal.Native.SafeChainEngineHandle
00007ffc2d69d528 2 64 Internal.Win32.SafeHandles.SafeRegistryHandle
00007ffc2d685bc8 2 64 Microsoft.Win32.SafeHandles.SafeWaitHandle
00007ffc2d685280 3 72 System.Threading.ThreadInt64PersistentCounter+ThreadLocalNodeFinalizationHelper
00007ffc2d5f5f50 3 72 System.Runtime.InteropServices.PosixSignalRegistration
00007ffc2d4299d0 1 72 Microsoft.Win32.SafeHandles.SafeFileHandle
00007ffc2d6e40b8 1 80 System.Runtime.Loader.DefaultAssemblyLoadContext
00007ffc2dac9ed0 2 96 PageIndex
00007ffc2d96d0c8 2 96 ColumnIndex
00007ffc2d464470 3 120 System.Gen2GcCallback
00007ffc2d40a620 1 120 System.IO.FileSystemWatcher
00007ffc2d96bc18 2 128 CellStore`1[[System.Int32, System.Private.CoreLib]]
00007ffc2dac20c8 2 144 System.Reflection.Emit.DynamicResolver
00007ffc2d680f10 3 144 System.Threading.LowLevelLock
00007ffc2d683c48 3 168 System.Threading.ThreadPoolWorkQueueThreadLocals
00007ffc2d681e80 1 176 System.Threading.LowLevelLifoSemaphore
00007ffc2dc25ef0 1 184 System.Collections.Concurrent.CDSCollectionETWBCLProvider
00007ffc2db8e658 1 184 System.Net.NetEventSource
00007ffc2db8c378 1 184 System.Net.NetEventSource
00007ffc2db38f90 1 184 System.Net.NetEventSource
00007ffc2d90c658 1 184 Microsoft.IO.RecyclableMemoryStreamManager+Events
00007ffc2d689b48 1 184 Microsoft.AspNetCore.Certificates.Generation.CertificateManager+CertificateManagerEventSource
00007ffc2d66f9f8 1 184 System.Diagnostics.Tracing.FrameworkEventSource
00007ffc2d66b720 1 184 System.Net.NetEventSource
00007ffc2d44d128 1 184 System.Buffers.ArrayPoolEventSource
00007ffc2d2e2ec8 1 184 System.Diagnostics.Tracing.NativeRuntimeEventSource
00007ffc2d694e10 1 192 System.Threading.Tasks.TplEventSource
00007ffc2d572ab0 1 192 Microsoft.Extensions.DependencyInjection.DependencyInjectionEventSource
00007ffc2d505f00 1 200 Microsoft.Extensions.Logging.EventSource.LoggingEventSource
00007ffc2db8ade8 1 224 System.Net.NameResolutionTelemetry
00007ffc2d428b08 7 224 System.Threading.PreAllocatedOverlapped
00007ffc2d563c78 1 232 System.Diagnostics.DiagnosticSourceEventSource
00007ffc2d61fe88 1 240 Microsoft.AspNetCore.Hosting.HostingEventSource
00007ffc2db6b788 8 256 System.Threading.TimerQueue+AppDomainTimerSafeHandle
00007ffc2d690270 1 280 System.Net.Sockets.SocketsTelemetry
00007ffc2db6bc80 1 296 System.Net.Http.HttpTelemetry
00007ffc2d68b998 1 336 Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelEventSource
00007ffc2dc21998 1 360 System.Net.Security.NetSecurityTelemetry
00007ffc2d2dae28 1 384 System.Diagnostics.Tracing.RuntimeEventSource
00007ffc2d66ad60 10 480 System.Net.Sockets.SafeSocketHandle
00007ffc2d2e0240 21 504 System.WeakReference`1[[System.Diagnostics.Tracing.EventSource, System.Private.CoreLib]]
00007ffc2d2b0538 9 648 System.Threading.Thread
00007ffc2d77a188 2 704 Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketReceiver
00007ffc2d90cec0 6 960 Microsoft.IO.RecyclableMemoryStream
00007ffc2d5fc658 10 1280 System.Net.Sockets.Socket
00007ffc2d68d898 4 1536 System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs
00007ffc2d2dc778 42 4704 System.Diagnostics.Tracing.EventSource+OverrideEventProvider
00007ffc2daec058 356 14240 System.Drawing.Bitmap
Total 553 objectsWOW！！！，看上面356個System.Drawing.Bitmap在等待回收，看起來這是我們的影響因素，我們來查一下程式碼。
try
{
cell.Value = string.Empty;
Bitmap bitmap;
if (url.IsBase64StringValid())
{
bitmap = url.Base64StringToBitmap();
}
else
{
bitmap = Extension.GetBitmapByUrl(url);
}
if (bitmap == null)
{
cell.Value = ExporterHeaderList[colIndex].ExportImageFieldAttribute.Alt;
}
else
{
ExcelPicture pic = CurrentExcelWorksheet.Drawings.AddPicture(Guid.NewGuid().ToString(), bitmap);
AddImage((rowIndex + (ExcelExporterSettings.HeaderRowIndex > 1 ? ExcelExporterSettings.HeaderRowIndex : 0)),
colIndex - ignoreCount, pic, ExporterHeaderList[colIndex].ExportImageFieldAttribute.YOffset, ExporterHeaderList[colIndex].ExportImageFieldAttribute.XOffset);
CurrentExcelWorksheet.Row(rowIndex + 1).Height = ExporterHeaderList[colIndex].ExportImageFieldAttribute.Height;
pic.SetSize(ExporterHeaderList[colIndex].ExportImageFieldAttribute.Width * 7, ExporterHeaderList[colIndex].ExportImageFieldAttribute.Height);
}
}
catch (Exception)
{
cell.Value = ExporterHeaderList[colIndex].ExportImageFieldAttribute.Alt;
}
在ExcelPicture物件中去使用Bitmap物件，對於線上圖片源來說，我們會讀取並儲存到Bitmap中，但是我們發現並沒有對該物件進行釋放操作，所以導致大量的Bitmap一直沒有釋放，我們通過using來處理一下。
using (ExcelPicture pic = CurrentExcelWorksheet.Drawings.AddPicture(Guid.NewGuid().ToString(), bitmap))
{
AddImage((rowIndex + (ExcelExporterSettings.HeaderRowIndex > 1 ? ExcelExporterSettings.HeaderRowIndex : 0)),
colIndex - ignoreCount, pic, ExporterHeaderList[colIndex].ExportImageFieldAttribute.YOffset, ExporterHeaderList[colIndex].ExportImageFieldAttribute.XOffset);
CurrentExcelWorksheet.Row(rowIndex + 1).Height = ExporterHeaderList[colIndex].ExportImageFieldAttribute.Height;
pic.SetSize(ExporterHeaderList[colIndex].ExportImageFieldAttribute.Width * 7, ExporterHeaderList[colIndex].ExportImageFieldAttribute.Height);
}一個帶有終結器的新物件是必須要被新增進finalization queue中的，這個行為也被稱為“終結註冊（registering for finalization）”。當然我也建議你選擇使用SOSEX擴充套件外掛，它提供了finalization類似的內容，似乎看起來更直觀一些，如下所示。
下載地址：http://www.stevestechspot.com...
:000> .load D:\sosex_64\sosex.dll
This dump has no SOSEX heap index.
The heap index makes searching for references and roots much faster.
To create a heap index, run !bhi
0:000> !finq -stat
Generation 0:
Count Total Size Type
---------------------------------------------------------
54 2160 System.Drawing.Bitmap
54 objects, 2,160 bytes
Generation 1:
Count Total Size Type
---------------------------------------------------------
1 184 Microsoft.AspNetCore.Certificates.Generation.CertificateManager+CertificateManagerEventSource
1 336 Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelEventSource
4 1536 System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs
1 32 Internal.Cryptography.Pal.Native.SafeCertStoreHandle
1 280 System.Net.Sockets.SocketsTelemetry
1 192 System.Threading.Tasks.TplEventSource
1 40 Internal.Cryptography.Pal.Native.SafeCertContextHandle
2 64 Internal.Win32.SafeHandles.SafeRegistryHandle
2 64 Internal.Cryptography.Pal.Native.SafeChainEngineHandle
1 32 Internal.Cryptography.Pal.Native.SafeLocalAllocHandle
1 80 System.Runtime.Loader.DefaultAssemblyLoadContext
1 24 System.WeakReference`1[[System.Runtime.Loader.AssemblyLoadContext, System.Private.CoreLib]]
1 24 System.WeakReference`1[[Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerOptions, Microsoft.AspNetCore.Server.Kestrel.Core]]
2 704 Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketReceiver
2 48 System.WeakReference`1[[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelConnection, Microsoft.AspNetCore.Server.Kestrel.Core]]
1 184 Microsoft.IO.RecyclableMemoryStreamManager+Events
6 960 Microsoft.IO.RecyclableMemoryStream
2 48 System.WeakReference`1[[System.Text.RegularExpressions.RegexReplacement, System.Text.RegularExpressions]]
1 64 CellStore`1[[OfficeOpenXml.ExcelCoreValue, Magicodes.IE.EPPlus]]
1 64 CellStore`1[[System.Object, System.Private.CoreLib]]
1 64 FlagCellStore
2 128 CellStore`1[[System.Int32, System.Private.CoreLib]]
1 64 CellStore`1[[System.Uri, System.Private.Uri]]
2 96 ColumnIndex
2 144 System.Reflection.Emit.DynamicResolver
1 24 System.WeakReference
2 96 PageIndex
302 12080 System.Drawing.Bitmap
1 184 System.Net.NetEventSource
1 40 Interop+WinHttp+SafeWinHttpHandle
8 256 System.Threading.TimerQueue+AppDomainTimerSafeHandle
1 296 System.Net.Http.HttpTelemetry
1 224 System.Net.NameResolutionTelemetry
1 184 System.Net.NetEventSource
1 184 System.Net.NetEventSource
1 360 System.Net.Security.NetSecurityTelemetry
1 24 System.Net.Security.SafeCredentialReference
1 184 System.Collections.Concurrent.CDSCollectionETWBCLProvider
1 48 System.Net.Security.SafeFreeCredential_SECURITY
1 32 Microsoft.Win32.SafeHandles.SafeBCryptAlgorithmHandle
499 objects, 30,736 bytes
Generation 2:
0 objects, 0 bytes
TOTAL: 553 objects, 32,896 bytes可能大家都會像我一開始有個疑問，你這個圖片我看了...沒有那麼大，並且在Windbg中也沒有表現大小呀。首先我們先來看一下這個圖片的質量。圖片的畫素為2560x1440，位深為24目前已知這些資訊，我們計算一下未壓縮的圖片大小。
2560x1440x24/8
10M左右一張圖，已知圖片數x10M=3G，其實對於這個問題來說，這並不屬於記憶體洩漏。
總結
這篇文章主要介紹如何利用Windbg分析應用程式中的記憶體問題，從託管堆到非託管堆的探索以及到記憶體的分配，最終根據記憶體的表現和理論確認記憶體的問題，當然對於記憶體分析建議大家不一定非要鍾情一個工具，當然可以結合著PerfView一起做也許效果更佳。
微軟最有價值專家（MVP）
微軟最有價值專家是微軟公司授予第三方技術專業人士的一個全球獎項。28年來，世界各地的技術社群領導者，因其線上上和線下的技術社群中分享專業知識和經驗而獲得此獎項。
MVP是經過嚴格挑選的專家團隊，他們代表著技術最精湛且最具智慧的人，是對社群投入極大的熱情並樂於助人的專家。MVP致力於通過演講、論壇問答、建立網站、撰寫部落格、分享視訊、開源專案、組織會議等方式來幫助他人，並最大程度地幫助微軟技術社群使用者使用Microsoft技術。
更多詳情請登入官方網站：
https://mvp.microsoft.com/zh-cn
歡迎關注微軟中國MSDN訂閱號，獲取更多最新發布！
