I was cleaning up one of my old programs and I got interesting warnings from code analysis:
CA1901 : Microsoft.Portability : As it is declared in your code, the return type of P/Invoke 'RichTextBoxEx.NativeMethods.SendMessage(IntPtr, Integer, Integer, Integer)' will be 4 bytes wide on 64-bit platforms. This is not correct, as the actual native declaration of this API indicates it should be 8 bytes wide on 64-bit platforms. Consult the MSDN Platform SDK documentation for help determining what data type should be used instead of 'Integer'.
CA1901 : Microsoft.Portability : As it is declared in your code, parameter 'wParam' of P/Invoke 'RichTextBoxEx.NativeMethods.SendMessage(IntPtr, Integer, Integer, Integer)' will be 4 bytes wide on 64-bit platforms. This is not correct, as the actual native declaration of this API indicates it should be 8 bytes wide on 64-bit platforms. Consult the MSDN Platform SDK documentation for help determining what data type should be used instead of 'Integer'.
Of course, I checked official API documentation but it wasn't too helpful. All it said was that return type is LRESULT and wParam is WPARAM. However, from message it was quite clear that I need something that is 8 bytes on 64-bit platform and 4 on 32-bit one.
I checked other code where I used that function and I noticed that I used IntPtr there. However, although this was technically correct, it didn't sound right. SendMessage has many purposes and here I was using it to set tab stops. While I could wrap number in IntPtr and pass it like that, I wanted another solution. Solution that will allow me to use number and still have IntPtr-like behavior.
Of course, as usual in .Net P/Interop, solution seemed quite simple. Use whatever data type you wish and just set correct marshaling. I used signature that P/Invoke Interop Assistant generated:
<DllImport("user32.dll", EntryPoint:="SendMessageW")> _
Public Shared Function SendMessageW(<InAttribute()> ByVal hWnd As IntPtr, ByVal Msg As UInteger, <MarshalAs(UnmanagedType.SysUInt)> ByVal wParam As UInteger, <MarshalAs(UnmanagedType.SysInt)> ByVal lParam As Integer) As <MarshalAs(UnmanagedType.SysInt)> Integer
End Function
On next run program greeted me with "Cannot marshal 'parameter #3': Invalid managed/unmanaged type combination (Int32/UInt32 must be paired with I4, U4, or Error).". No matter what I did, it would not budge. It seems that this tool disappointed me once again.
To keep long story short, this is final function I decided to use:
<DllImport("user32.dll", EntryPoint:="SendMessageW", CharSet:=CharSet.Unicode)> _
Friend Shared Function SendMessageW(ByVal hwnd As IntPtr, ByVal Msg As UInteger, ByVal wParam As IntPtr, ByRef lParam As IntPtr) As IntPtr
End Function
Since my code still needed to pass integers I made wrapper to sort that out:
Public Shared Function SendMessage(ByVal hWnd As IntPtr, ByVal Msg As UInteger, ByVal wParam As Integer, ByVal lParam As Integer) As IntPtr
Return SendMessageW(hWnd, Msg, New IntPtr(wParam), New IntPtr(lParam))
End Function
With that my work was done.
P.S. Yes, you are not mistaken, code is in VB.NET. It is program I wrote a while back and I have no intention to rewrite it in C# just to get single bug fixed.
P.P.S. I find it interesting that LRESULT is really IntPtr although name does not suggest so (at least not to me). HRESULT on other hand is not IntPtr although from name I would deduce it is handle (and thus IntPtr).