Hosting ActiveX control in a Win32 app using ATL ends up in a crash

Issue

Recently had a customer who was facing this issue. His code looked as follows (assuming COM’s initialized) which apparently is trying to setup an ActiveX control…

HWND hWnd = CreateWindow(_T(ATLAXWIN_CLASS),
_T(“SysMon”),
WS_CHILD,
0, 10, 100, 200,
hwndParent,
NULL,
hInstance,
NULL);

As soon as above code executes the application crashes.

Why loading ActiveX failed?

Callstack for the crash looks as follows…

ConsoleApplication1.exe!ATL::CComPolyObject<ATL::CAxHostWindow>::CComPolyObject<ATL::CAxHostWindow>
ConsoleApplication1.exe!ATL::CComCreator<ATL::CComPolyObject<ATL::CAxHostWindow> >::CreateInstance
ConsoleApplication1.exe!AtlAxCreateControlLicEx
ConsoleApplication1.exe!AtlAxCreateControlLic
ConsoleApplication1.exe!ATL::AtlAxWindowProc
user32.dll!UserCallWinProcCheckWow
user32.dll!DispatchClientMessage
user32.dll!__fnINLPCREATESTRUCT
ntdll.dll!KiUserCallbackDispatch    Unknown
user32.dll!ZwUserCreateWindowEx    Unknown
user32.dll!VerNtUserCreateWindowEx
user32.dll!CreateWindowInternal
user32.dll!CreateWindowExW
ConsoleApplication1.exe!wmain
ConsoleApplication1.exe!__tmainCRTStartup
ConsoleApplication1.exe!wmainCRTStartup
kernel32.dll!BaseThreadInitThunk
ntdll.dll!RtlUserThreadStart

 

Autos debug window showed following…

Resolution

The highlighted elements gave ample hint. This is a console application with no ATL boilerplate code generated by project wizard for this project. Ideally ATL requires a global CComModule instance to initialize itself. This is how the CComModule constructor looks like…

CComModule()
{
// Should have only one instance of a class
// derived from CComModule in a project.
ATLASSERT(_pModule == NULL);
<span style="background-color: #ffff00;">_pModule = this;</span>
#if !defined(_ATL_NATIVE_INITIALIZATION)
#pragma warning(push)  // disable 4483
#pragma warning(disable:4483)
using namespace __identifier("<AtlImplementationDetails>");
#pragma warning(pop)  // disable 4483
ATLASSERT(ComModuleInitialized == false);
// If ComModuleHelper.Module == NULL it mean that DllMain has not been called, so we assume CComModule lives in
// an exe and not in a dll
if (ComModuleHelper.Module != NULL)
{
ATLASSERT(ComModuleHelper.Module == this);
_DllMain(ComModuleHelper.Instance, DLL_PROCESS_ATTACH, NULL, ComModuleHelper.ObjectMap, ComModuleHelper.LibraryId);
}
ComModuleInitialized = true;
#endif
}

This function further initializes _pModule instance to point to this global instance of ours eventually. So after we declare CComModule global instance this is how _pAtlModule (it points to our  global instance) will look after instantiation of our global CComModule…

Crash resolved.

GetDlgItem fails for an ActiveX control

Was working with a customer yesterday who was having this issue of GetDlgItem failing to return a valid window handle for an ActiveX control placed on a dialog. The ActiveX control is visible though but curiously the handle was always showing up as NULL.

While debugging saw this particular line in the Visual Studio 2010 output window…

Hmm interesting. Why is the control saying I want to be windowless. Customer never mentioned he wants a windowless control.

So when creating an ActiveX control the wizard provides a checkbox which says the following…

When the above checkbox is checked the wizard adds the following line of code into your ActiveX control’s constructor…

In my customer’s case this was the reason for GetDlgItem not returning a valid window object for the ActiveX control.

Another way this problem can manifest is when you turn off m_bWindowOnly in your activex control class. This variable will be off if you are creating a windowless control.

In short I would say: always keep an eye on the visual studio output window. 🙂

ATL 9.0 displayed in the middle of an Activex control

Recently a customer of mine faced this issue. So he had an ActiveX control and when displaying the ActiveX control in browser a string is displayed right in the middle of the control: “ATL 9.0”.

So this issue happens because you didn’t override CComControlBase::OnDraw  function. The default code for CComControlBase::OnDraw looks like the following…

HRESULT CComControlBase::YourClassName::OnDraw(_In_ ATL_DRAWINFO& di)
{
  ::SetTextAlign(di.hdcDraw, TA_CENTER|TA_BASELINE);
  LPCTSTR pszText = _T("ATL ") _T(_ATL_VER_RBLD); // "ATL 9.0"
  ::TextOut(di.hdcDraw,
            di.prcBounds->left + (di.prcBounds->right - di.prcBounds->left) / 2,
            di.prcBounds->top + (di.prcBounds->bottom - di.prcBounds->top) / 2,
            pszText,
            lstrlen(pszText)); 
   return S_OK;
}

MSDN documentation for CComControlBase::OnDraw function confirms this behavior. Quote…

The wizard’s default OnDraw draws a rectangle with the label “ATL 8.0”.

The solution for this is to override CComControlBase::OnDraw function in your derived class and provide your own drawing or just return S_OK.

Set resource handle in MFC and ATL

When working in MFC/ATL and then using a resource dll can at times cause bugs because resource handle is not properly set. For e.g. if you have a dialog resource in a dll and you instantiate a CDialog which results in an error because resource cannot be located in this application or dll since it’s in the resource dll. So solution for above problem will be to set correct resource handle.

MFC provides two functions…

1. AfxGetResourceHandle
2. AfxSetResourceHandle

pretty obvious names. 🙂

So we should first save our current resource handle and then set new resource handle before loading such a resource. Also don’t forget to set our old handle since its good practice.

Sample code snippet.

extern HINSTANCE hResDll = NULL;
HINSTANCE hDefInstance = AfxGetResourceHandle();
AfxSetResourceHandle(hResDll);
CBitmap Bmp;
Bmp.LoadBitmap( IDB_NIBUS_FACE );

AfxSetResourceHandle(hDefInstance);

Internally MFC calls FindResource and LoadResource using this handle, so if it’s not properly set this will cause failure in loading resources.

In ATL it’s quite similar except that we’ve got a new function called AddResourceInstance which adds our new resource handle to existing list of handles. So when a look up is done given resource handle is also used. Following functions are provided in ATL to work with resource handles…

1. AddResourceInstance
2. GetResourceInstance – Similar to AfxGetResourceHandle
3. SetResourceInstance – Similar to AfxGetSetResourceHandle

For newbies this is always a painful bug to resolve as they don’t know what went wrong since they expect this to be automagically done.

How to convert ANSI string to UNICODE string and vice versa?

Well quite simple,  but still quite frequently asked in forums… 🙂

There are two macros that does this for us. They are as follows.

Note: You must include atlconv.h

A2W – ANSI to UNICODE
W2A – UNICODE to ANSI

Before using these two macros you have to use this macro too…

USES_CONVERSION

Here is a code snippet for you… 😉

#include <atlconv .h>
//An example for converting from ANSI to UNICODE

//use this first
USES_CONVERSION;

//An ANSI string
LPSTR lpsz_ANSI_String = "An ANSI String";

//ANSI string being converted to a UNICODE string
LPWSTR lpUnicodeStr = A2W( lpsz_ANSI_String )

//Another example for converting from UNICODE to ANSI

//Use this first
USES_CONVERSION

//A UNICODE string
LPWSTR lp_UNICODE_STR = L"A Unicode String";

//UNICODE string being converted to a ANSI string
LPSTR  lpsz_ANSI_STR = W2A( lp_UNICODE_STR );

Another option is to call WideCharToMultiByte and MultiByteToWideChar directly, it’s quite easy to use, at least easier than above macros. If you have doubts on usage of above functions then take a look at AfxW2AHelper and AfxA2WHelper functions. Since these are the functions that A2W and W2A internally calls.

Also as a homework take a look at CW2AEX and CA2WEX classes. Will be similar to above macros, looks to me like a  secure version overload.