January 8, 2015 at 15:27We remove the dump of objects from the memory .Net application
We continue the topic of interesting things on .Net, from which the Java world will chuckle (although it is also possible for them to do this), and C ++ adherents say: “what they just won’t do in order not to learn C ++”. In this note, we will write in essence - a simple kernel of the memory profiler for the .Net platform, which will remove the dump from the SOH heap (and in the future with LOH).
To write an article, we need the code from the article
Getting a pointer to an .Net object and Manual cloning of a stream (measuring the size of objects). Our goals for today:
- Learn to iterate a bunch of .Net
- Learn to find the beginning of the .Net heap
- Try to ster all the objects of another domain.
Link to the project on GitHub:
DotNetExFor a start, a full list of articles posted on Habré for this cycle
Making shipped assemblies: interacting between domains without marshalling.
Obtaining a .Net object pointer.
Manual cloning of a stream. When the Assembler + C # or Java = Love
Changing code system assemblies or "leakage» .Net Framework 5.0
How does decompilation in .Net or Java to .Net example,
continue to shred CLR: .Net objects outside the pool heaps SOH / LOH
Remove objects from a memory dump .Net applications
Obtaining a .Net object pointer.
Manual cloning of a stream. When the Assembler + C # or Java = Love
Changing code system assemblies or "leakage» .Net Framework 5.0
How does decompilation in .Net or Java to .Net example,
continue to shred CLR: .Net objects outside the pool heaps SOH / LOH
Remove objects from a memory dump .Net applications
As last time, we will solve problems as they become available:
- Find the beginning of the heap in .Net
As we probably know, there are two kinds of heaps in .Net. This is a bunch for small objects and a bunch for large objects (> 85K). They differ primarily in the organization of objects within themselves. If in SOH objects are allocated one after another, then in LOH everything is based on linked lists and a table of free spaces between occupied sections. This is what we know. But the truth is different. Firstly, SOH cannot be continuous due to the possibility of stumbling onto a busy portion of virtual memory while expanding it. The second - in fact, that objects are continuously highlighted one after another is a beautiful assumption, because there are pinned objects that cannot be moved, which means that empty spaces will appear when the heap is compressed. This means that SOH also contains a void table. And as a result, this should mean specifically for us that:
To simplify the example, let's only dump the objects from our heap. In order to do this, we will take a pointer to any object and using the WinApi VirtualQuery function, we will try to get the region of virtual memory that was allocated for this heappublic static void GetManagedHeap(out IntPtr heapsOffset, out IntPtr lastHeapByte) { // получаем указатель на любой объект. Для этого просто выделим его var offset = EntityPtr.ToPointer(new object()); var memoryBasicInformation = new WinApi.MEMORY_BASIC_INFORMATION(); unsafe { WinApi.VirtualQuery(offset, ref memoryBasicInformation, (IntPtr)Marshal.SizeOf(memoryBasicInformation)); heapsOffset = (IntPtr)memoryBasicInformation.AllocationBase; lastHeapByte = (IntPtr)((long)offset + (long)memoryBasicInformation.RegionSize); } }
Perfectly! Now we know the piece of memory on which to search for objects. - We ster all the objects of a foreign domain
In order to do this, we need to understand how to recognize them. So, if we sit with the debugger and study the memory, we can finally conclude that the reference to the object does not point to the first field of the object, but to the pointer to the table of virtual methods. Which is actually not only a table of virtual methods, but methods in general and type descriptions. After all, we are not in the C ++ world, but in the .Net world, where for each object you can understand, "whose will you be?" But what about SyncBlockIndex, you ask? And he, as it turned out, is “in front” of the object, minus the size of the word (4 bytes on the 32nd and 8 bytes on the 64-bit system). Therefore, the title structure of any object is as follows:[StructLayout(LayoutKind.Explicit)] public unsafe struct EntityInfo { [FieldOffset(0)] public int SyncBlockIndex; [FieldOffset(4)] public MethodTableInfo *MethodTable; } [StructLayout(LayoutKind.Explicit)] public struct RefTypeInfo { [FieldOffset(0)] public EntityInfo BasicInfo; [FieldOffset(8)] public byte fieldsStart; }
Next, let's see what MethodTableInfo looks like:[StructLayout(LayoutKind.Explicit)] public unsafe struct MethodTableInfo { #region Basic Type Info [FieldOffset(0)] public MethodTableFlags Flags; [FieldOffset(4)] public int Size; [FieldOffset(8)] public short AdditionalFlags; [FieldOffset(10)] public short MethodsCount; [FieldOffset(12)] public short VirtMethodsCount; [FieldOffset(14)] public short InterfacesCount; [FieldOffset(16)] public MethodTableInfo *ParentTable; #endregion [FieldOffset(20)] public ObjectTypeInfo *ModuleInfo; [FieldOffset(24)] public ObjectTypeInfo *EEClass; }
Here is only part of all the information from this structure. In fact, it is more extensive. The EEClass field is important here, which leads to the structure of the type description. I practically did not study it. The content looks something like this:[StructLayout(LayoutKind.Explicit)] public unsafe struct ObjectTypeInfo { [FieldOffset(0)] public ObjectTypeInfo *ParentClass; [FieldOffset(16)] public MethodTableInfo *MethodsTable; }
Since only the MethodsTable field is important to us, I just found it. I missed the rest. Why do we need it? This is a reference to MethodsTable, which is referenced here by its EEClass field.
In this way, not in a cunning way, we found a somewhat inaccurate, but perfectly working method for detecting a .Net object. It looks like this:private static unsafe bool IsCorrectMethodsTable(IntPtr mt) { if (mt == IntPtr.Zero) return false; if (PointsToAllocated(mt)) if (PointsToAllocated((IntPtr) ((MethodTableInfo*) mt)->EEClass)) if (PointsToAllocated((IntPtr) ((MethodTableInfo*) mt)->EEClass->MethodsTable)) return ((IntPtr) ((MethodTableInfo*) mt)->EEClass->MethodsTable == mt) || ((IntPtr) ((MethodTableInfo*) mt)->ModuleInfo == MscorlibModule); return false; } private static bool PointsToAllocated(IntPtr ptr) { if (ptr == IntPtr.Zero) return false; return !WinApi.IsBadReadPtr(ptr, 32); }
For each pointer, it is checked whether it points to the allocated memory. This is the first barrier. The second barrier is if the first field of a theoretical object indicates something that we initially interpret as MethodsTable. We check that the EEClass field points to the allocated memory and interpret this pointer as a pointer to the ObjectTypeInfo structure, then check if the pointer to the MethodsTable is equal to our found pointer (is there a back link?) If everything is normal, the object is found.
It remains only to walk through all areas of memory and try to recognize objects there. I will not post this sheet, because there is also a code for registering what is found and displaying it on the screen. I’ll just say that the problem is solved, the link to exit the program is below.
Link to the full example code:GitHub / DotNetEx / AdvSampleThe output of our program, a memory dump00606 : System.String 00583 : System.Object 00277 : System.RuntimeType 00072 : System.Array+SZArrayEnumerator 00046 : System.Char[] 00041 : System.Int32[] 00033 : System.String[] 00032 : System.Object[] 00030 : System.Version 00029 : System.Byte[] 00024 : System.Text.StringBuilder 00023 : System.Collections.Hashtable+bucket[] 00020 : System.Security.PermissionSet 00020 : System.Collections.Hashtable 00020 : System.Reflection.AssemblyName 00014 : System.Reflection.RuntimeAssembly 00014 : System.Security.Permissions.EnvironmentPermission 00013 : System.Collections.Hashtable+SyncHashtable 00012 : System.Globalization.CompareInfo 00012 : System.Collections.Generic.Dictionary`2+Entry[[System.Type, mscorlib, Ve rsion=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Securit y.Policy.EvidenceTypeDescriptor, mscorlib, Version=4.0.0.0, Culture=neutral, Pub licKeyToken=b77a5c561934e089]][] 00011 : System.Globalization.CultureInfo 00010 : System.Collections.ArrayList 00010 : System.Int32 00010 : System.Threading.ThreadStart 00010 : System.Internal.HandleCollector+HandleType 00010 : System.Internal.HandleCollector+HandleType 00009 : System.Security.Permissions.SecurityPermission 00009 : System.EventHandler 00009 : Microsoft.Win32.SafeHandles.SafeRegistryHandle 00009 : Microsoft.Win32.RegistryKey 00008 : System.Threading.Thread 00008 : Microsoft.Win32.SafeHandles.SafeWaitHandle 00008 : System.Security.Policy.EvidenceTypeDescriptor 00008 : System.Runtime.InteropServices.HandleRef 00008 : System.UInt16 00006 : System.Runtime.Remoting.Metadata.SoapTypeAttribute[] 00005 : System.Type[] 00005 : System.Threading.ReaderWriterLock 00005 : System.Reflection.CustomAttributeRecord[] 00005 : System.Globalization.TextInfo 00005 : System.Security.Permissions.EnvironmentStringExpressionSet 00005 : System.WeakReference 00005 : System.Threading.ThreadHelper 00004 : System.Security.FrameSecurityDescriptor 00004 : System.Reflection.RuntimeModule 00004 : System.Reflection.RuntimeConstructorInfo 00004 : System.Guid 00004 : Microsoft.Win32.SafeHandles.SafePEFileHandle 00004 : System.Security.Policy.Evidence 00004 : System.Collections.Generic.Dictionary`2[[System.Type, mscorlib, Version= 4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Security.Poli cy.EvidenceTypeDescriptor, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKey Token=b77a5c561934e089]] 00004 : System.Security.Policy.AssemblyEvidenceFactory 00004 : System.Security.Policy.Evidence+EvidenceUpgradeLockHolder 00003 : System.Security.Util.TokenBasedSet 00003 : System.Globalization.CultureData 00003 : System.Reflection.MemberFilter 00003 : System.Reflection.MethodBase[] 00003 : System.RuntimeType[] 00003 : System.Runtime.Remoting.Lifetime.LeaseLifeTimeServiceProperty 00003 : System.Attribute[] 00003 : System.Threading.ManualResetEvent 00003 : System.IO.PathHelper 00003 : System.Security.Permissions.UIPermission 00002 : System.AppDomainSetup 00002 : System.Security.PermissionToken 00002 : System.Runtime.Remoting.Contexts.IContextProperty[] 00002 : System.Reflection.TypeFilter 00002 : System.Collections.Queue 00002 : System.WeakReference[] 00002 : System.Char 00002 : System.Security.Policy.StrongName[] 00002 : System.Reflection.RuntimeMethodInfo 00002 : System.Threading.SynchronizationContext 00002 : System.Internal.HandleCollector+HandleType[] 00002 : System.Globalization.NumberFormatInfo 00002 : Microsoft.Win32.SystemEvents+SystemEventInvokeInfo[] 00002 : System.Text.EncoderReplacementFallback 00002 : System.IO.UnmanagedMemoryStream 00002 : System.Security.Permissions.FileIOAccess 00002 : System.Security.Util.StringExpressionSet 00001 : System.Exception 00001 : System.OutOfMemoryException 00001 : System.StackOverflowException 00001 : System.ExecutionEngineException 00001 : System.AppDomain 00001 : System.Security.PermissionTokenFactory 00001 : System.Security.PermissionToken[] 00001 : System.Globalization.CalendarData[] 00001 : System.Globalization.CalendarData 00001 : System.__Filters 00001 : System.DefaultBinder 00001 : System.RuntimeType+RuntimeTypeCache+MemberInfoCache`1[[System.Reflection .RuntimeConstructorInfo, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyTo ken=b77a5c561934e089]] 00001 : System.Reflection.RuntimeConstructorInfo[] 00001 : System.Reflection.ConstructorInfo[] 00001 : System.Collections.Generic.List`1[[System.Reflection.MethodBase, mscorli b, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] 00001 : System.Signature 00001 : System.Reflection.ParameterInfo[] 00001 : System.Int32[][] 00001 : System.Runtime.Remoting.Proxies.ProxyAttribute 00001 : System.Runtime.Remoting.DomainSpecificRemotingData 00001 : System.Runtime.Remoting.Channels.ChannelServicesData 00001 : System.Runtime.Remoting.Activation.LocalActivator 00001 : System.Runtime.Remoting.Activation.ActivationListener 00001 : System.Runtime.Remoting.Contexts.ContextAttribute[] 00001 : System.Runtime.Remoting.Contexts.Context 00001 : System.Runtime.Remoting.Messaging.ConstructorCallMessage 00001 : System.Runtime.Remoting.Metadata.RemotingTypeCachedData 00001 : System.Collections.Generic.Dictionary`2[[System.RuntimeType, mscorlib, V ersion=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Runtim eType, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e0 89]] 00001 : System.Collections.Generic.Dictionary`2+Entry[[System.RuntimeType, mscor lib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System. RuntimeType, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c56 1934e089]][] 00001 : System.AttributeUsageAttribute 00001 : System.Runtime.Remoting.Metadata.SoapTypeAttribute 00001 : System.Runtime.Remoting.Activation.ConstructionLevelActivator 00001 : System.Runtime.Remoting.RemotingConfigHandler+RemotingConfigInfo 00001 : System.Collections.Generic.Dictionary`2[[System.String, mscorlib, Versio n=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Globalizati on.CultureData, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5 c561934e089]] 00001 : System.Runtime.Remoting.ObjectHandle 00001 : System.Diagnostics.TraceSwitch 00001 : System.Collections.Generic.Dictionary`2[[System.Int16, mscorlib, Version =4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.IntPtr, msco rlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] 00001 : System.Collections.Generic.GenericEqualityComparer`1[[System.Int16, msco rlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] 00001 : System.Collections.Generic.ObjectEqualityComparer`1[[System.IntPtr, msco rlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] 00001 : System.Threading.Mutex 00001 : System.Runtime.CompilerServices.RuntimeHelpers+CleanupCode 00001 : System.Threading.Mutex+MutexCleanupInfo 00001 : System.Threading.Mutex+MutexTryCodeHelper 00001 : System.Threading.EventWaitHandle 00001 : System.Threading.HostExecutionContextManager 00001 : System.Collections.Generic.ObjectEqualityComparer`1[[System.Type, mscorl ib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] 00001 : System.Security.Policy.ApplicationTrust 00001 : System.Security.Policy.PolicyStatement 00001 : System.Collections.Generic.List`1[[System.Security.Policy.StrongName, ms corlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] 00001 : System.SZArrayHelper+SZGenericArrayEnumerator`1[[System.Security.Policy. StrongName, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561 934e089]] 00001 : System.Collections.ObjectModel.ReadOnlyCollection`1[[System.Security.Pol icy.StrongName, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5 c561934e089]] 00001 : Microsoft.Win32.Win32Native+OSVERSIONINFO 00001 : Microsoft.Win32.Win32Native+OSVERSIONINFOEX 00001 : System.OperatingSystem 00001 : System.__ComObject 00001 : System.Collections.Queue+SynchronizedQueue 00001 : System.Threading.AutoResetEvent 00001 : System.Threading.ContextCallback 00001 : System.RuntimeMethodInfoStub 00001 : System.RuntimeType+RuntimeTypeCache+MemberInfoCache`1[[System.Reflection .RuntimeMethodInfo, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b 77a5c561934e089]] 00001 : System.Reflection.RuntimeMethodInfo[] 00001 : System.Collections.Generic.List`1[[System.Attribute, mscorlib, Version=4 .0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] 00001 : System.Drawing.SizeF 00001 : System.Drawing.Point 00001 : System.Windows.Forms.Application+ThreadContext 00001 : System.Windows.Forms.WindowsFormsSynchronizationContext 00001 : System.EventArgs 00001 : System.Windows.Forms.NativeMethods+WNDCLASS_D 00001 : Microsoft.Win32.UserPreferenceChangedEventHandler 00001 : System.Random 00001 : System.Collections.Generic.ObjectEqualityComparer`1[[System.Object, msco rlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] 00001 : System.Collections.Generic.Dictionary`2[[System.String, mscorlib, Versio n=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Object[], m scorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] 00001 : Microsoft.Win32.SystemEvents 00001 : System.Runtime.Remoting.Messaging.LogicalCallContext 00001 : System.Text.UTF8Encoding 00001 : Microsoft.Win32.NativeMethods+WNDCLASS 00001 : System.Internal.HandleCollector+HandleType[] 00001 : System.IntPtr[] 00001 : System.Security.Util.URLString 00001 : System.Security.Permissions.FileIOPermission 00001 : System.Security.Util.LocalSiteString 00001 : System.Security.Util.DirectoryString 00001 : Microsoft.Win32.SafeHandles.SafeFileHandle 00001 : System.Text.SBCSCodePageEncoding 00001 : System.Text.InternalEncoderBestFitFallback 00001 : System.Text.InternalDecoderBestFitFallback 00001 : Microsoft.Win32.SafeHandles.SafeViewOfFileHandle 00001 : Microsoft.Win32.SafeHandles.SafeFileMappingHandle 00001 : System.IO.__ConsoleStream 00001 : System.Text.EncoderNLS 00001 : System.IO.StreamWriter+MdaHelper 00001 : System.IO.TextWriter+SyncTextWriter 00001 : System.Diagnostics.Stopwatch 00001 : System.Predicate`1[[System.Int64, mscorlib, Version=4.0.0.0, Culture=neu tral, PublicKeyToken=b77a5c561934e089]] 00001 : System.DBNull Objects total: 2294. Time taken: 437 - Can I get objects from someone else's domain?
Of course! After all, we are looking at virtual memory. This time ... And the second ... there are no borders between domains, objects stand out one after another even when crossing the domain border. The difference is in the code. Therefore, for example, you can transfer an object without serialization between domains.
