Archive

Archive for the ‘Debugging’ Category

Breakpoints in Windbg

January 15, 2011 Leave a comment

WinDbg¬†rocks. ūüôā Setting breakpoints¬†is very easy in WinDbg. The command to set a breakpoint¬†is ‘bp’. So if you want to break whenever a¬†dll¬†is loaded into a process then type in following command…

bp kernel32!LoadLibraryW

So to trigger this breakpoint¬†attach ‘notepad.exe’ to the debugger and then type in this command. Now let the app run (press F5). Goto File->Open (this will trigger a definite LoadLibrary¬†:)). Now have a look¬†in WinDbg which¬†will have following output…

Breakpoint 0 hit
kernel32!LoadLibraryW:
00000000`76e50420 4533c0          xor     r8d,r8d

¬†To view call stack, type in ‘kpn’. I’ll blog more on breakpoints as and when I get time. Happy debugging. ūüôā

Advertisements

Breakpoints in Visual Studio

May 2, 2009 4 comments

What’s a breakpoint?

A breakpoint is defined as the location where a debugger breaks execution to allow the user to have a look or to modify the execution context.

What’s new with breakpoints?

With visual studio 2005 and 2008 behavior of breakpoint has changed. Some features that were added are as follows…

  1. Know hit count of a break point, no more need to keep a temp debugger variable for counting hits. For e.g. you can set a conditional breakpoint and then enable hit count. You’ll see how many times the condition was satisfied, you can also disable breaking of execution so that the program keeps running.
  2. Trace local variables/function name and more to the visual studio immediate/output window.
  3. Run a macro when a breakpoint is hit.
  4. We can disable breaking of execution which means there will only be tracing going on and no breaking of execution.

So to access these features, after adding a breakpoint, open the breakpoint window and right click on this breakpoint. Lower half of the dropdown contains these features, here is a screenshot of the context menu…

Breakpoint Context Menu

Breakpoint Context Menu

There are different types of break points available in visual studio. There are four in my knowledge…

  1. File breakpoint – Breaks at a location in a file
  2. Address breakpoint – Breaks at an address
  3. Function breakpoint – Breaks at a function
  4. Data breakpoint – Breaks at specific byte locations

Hit count

Let me show you how to count the number of even numbers from 1 to 100, I know it’s quite easy, it’s fifty, but via debugger? Here is a small function with a loop from 1 to 100.

void CountEven( const int From, const int To )
{
   for( int Index = From; Index < To; ++Index )
   {
      // Set a conditional breakpoint to get the count of odd numbers,
      // Dummy code to allow set a breakpoint
      ::SendMessage( AfxGetMainWnd()->GetSafeHwnd(), WM_NULL, 0, 0 );
   }
}

Now set a break point inside the for loop (Press F9) and right click and select “Breakpoint->Condition” item. You’ll get the following dialog…

Setting a Conditional breakpoint

Setting a Conditional breakpoint

So here I’ve set the condition that whenever Index%2 is zero then we have an even number. So when this happens I’m asking the debugger to break execution. So this works as expected but our aim is to count the number of even numbers using debugger, so for that again right click on the breakpoint line and select “Breakpoint->Hit Count” item, following dialog pops up…

Enable hit count in the debugger

Enable hit count in the debugger

You can see that I’ve selected an option called “break when the hit count is a multiple of”. I’ve given 10 as the option, so that frequency of breaking of execution is less. When I run the code this is what I get in the breakpoints window…

Hit count result

Hit count result

See the “Hit Count” item?¬† Execution broke 5 times this means for every 10 hits, hence count is 50. Isn’t this fun?

Tracing

Remember the times when we had to trace statements just for debugging purpose, for e.g. we wanted to trace a certain variable’s value when it satisfies a condition. Let’s take the above example, we’ve already enabled a conditional breakpoint such that it’s breaks execution whenever “Index” is an even number. Right click on the breakpoint line and select “Breakpoint” and then “When Hit” option. Following dialog pops up…

When Hit Dialog

When Hit Dialog

Check the first option Print a message”, this is the option that prints a message either to your “immediate window” or to your “Output window” based on the options that you’ve set. Now run the program to have some fun, this is the output in my computer…

Output of "when hit"

Output of "when hit"

All even index values are traced to the output window, isn’t this cool?

Filter

This one is even more cool. It allows us to set breakpoint filters. This is kind of a conditional but with a larger scope. For e.g. if you want to break execution only for a certain thread or only for a certain process. So for this right click on the breakpoint line and select “Breakpionts->Filter”. Following dialog pops up…

Breakpoint filter

Breakpoint filter

Pretty easy to use. So this means for a function that is called from multiple threads you can explicitly tell the debugger for which threads to break execution.

Tid bits

  1. Did you know that you can set a breakpoint in the call stack window by press F9?
  2. Did you know that you can delete all breakpoints by press Ctrl + Shift + F9?
  3. Did you know that you can run to cursor by pressing Ctrl + F10?
  4. Did you know that you display breakpoints window by pressing Alt + F9?
  5. Did you know that you can use “Set Next Statement” by pressing Ctrl + Shift + F10?

Yeap that’s it from me for now, all the best. ūüôā

Debugging WM_PAINT messages

November 15, 2008 1 comment

It’s always irritating to debug paint messages and sometimes expensive too. I’ve tried different methods to try debug painting issues, always ended up disappointed.

Why is it such a pain to debug WM_PAINT messages?

Main reason is, we are not in control. Paint messages comes via a window framework. Painting is done whenever a window gains focus or is activated this means our application will never get focus if we set a breakpoint in an OnPaint function because as soon as our application comes to focus it again sends a repaint request which again breaks into the OnPaint function.

It’s can be expensive too because we’ll need some kind of a dual monitor setup. So that our application runs on one monitor and the debugger on the other. For professional game/UI developers it’s ok because that’s their job and they will always need such a setup.¬†But guys like me cannot affort such a setup.

Any workaround?

Yes, that’s the good news. Windows explorer has a cute feature called “Tile Horizontally” and “Tile Vertically”. It also has another feature to select taskbar items using control key.

So this is how a standar taskbar looks, as you already know that at a time, in windows, only one window can be active.

taskbar_oneapp_selected

The tab having a dark shade is the one that’s active, and if you right click on this tab you’ll get a context menu which will correspond to the active application’s system menu.

Now here is the interesting part use ‘Ctrl’ key to select multiple tabs. Press ‘Ctrl’ and then use your mouse to click on tabs you wanna select and then right click on any of these selected tabs and see what you get! Here is a screenshot of what I get…

taskbar_multipleapp_selected

These menu items are interesting, most windows users are unaware of this option, it’s a quick way to arrange your windows. You can try them one by one. Note that there is an option to close a group too. ūüôā

So returning back to our original discussion, we now apply this trick to our application. We are going to select visual studio and an application that’s being¬†debugged.¬†I’ve set a break point in the OnPaint function.

So we select both applications in the taskbar using ‘Ctrl’ key and mouse and then right click and select ‘Tile vertically’ or ‘Tile Horizontally’. I would select ‘Tile vertically’ as it would give me enough vertical space but you can decide what’s best for you. Eventually what I get is a screen like this with two windows that doesn’t overlap each other. One of them is visual studio and the other one is my application that’s being debugged, also note that I’ve set a breakpoint in OnPaint function. Both applications behave independently without interfering with each other.

vertical_tile

Now my desktop acts like a dual monitor setup. ūüėČ It’s kinda cool for debugging purpose and yeah I agree this is not the ultimate solution but it’s quite useful. Hope this helps you. ūüôā

How to debug a dll?

October 25, 2008 5 comments

Why?

Well debugging exe is quite easy, but ever wondered how to debug a dll. Well why? Simple, you write a dll and you wanna debug a bug just like you would do with an executable.

Can you give us some examples?

Well so what is a dll? I think of it as a parasite because just like a parasite it’s dependent on an executable to get it running. Some examples follow…

  1. A good example to think of will be rundll32.exe. Ever thought about that name “rundll32.exe”, it just means “run a 32 bit dll using this exe”, so how does this achieve this, remember the command line that we pass to it, i.e. name of the dll followed by a procedure name exported from our dll.
  2. Another example to think of is regsvr32.exe, does the same stuff i.e. registers a COM dll¬†but this time with a difference, we don’t have to give a procedure name. Because COM programming protocol defines mandatory functions to be exported from a COM dll and regsvr32.exe knows exactly which function to call based on the command line switches passed in.
  3. Another example that comes to mind is the¬†windows surrogate process “dllhost.exe”, I’ve heard people say that it’s a virus ;). Naah it’s a COM surrogate process which helps in hosting a COM dll to make the dll behave like a process hence providing the flexibility to behave like an executabable and at the same time like a dll, vice versa is not possible. Well it can at times be hosting a malicious dll too. It’s quite rare though.

What has this got to do with debugging a DLL?

The reason why I’ve given above examples is to help you understand that a DLL cannot run by itself, it needs a host process to help it run.

Exactly! same for debugging it needs a process to get it running so that execution reaches our breakpoint that we’ve set. Aha! so there must be some option to tell the debugger about the host process. Oh yeah, goto Project settings->Debugging->Executable for Debug Session. Select any process which is using your dll.

Think that you are writing a window’s shell extension, which are dlls, think on how to debug them. So for this case your target process will be explorer.exe. So you will specify explorer.exe as your process. Or imagine writing an¬†activex control, so for this case the target process will be an activex test container.

This is a shot from VC6, it’s quite similar in VC7, VC8 and VC9, instead of tabs there you have a tree menu holding¬†the “Debugging” item right at the top of the tree.

Well now just press F5 and see what happens, you will see your target process running and just try doing a button or menu click that could invoke the function in which you’ve set a breakpoint.¬† If you are writing a shell extension for e.g. extending shell context menu, then you would right click on a file in the explorer window.

Also note that if your active project is a ‘dll project’ then you’ll get prompted to enter a target process name to get the dll hosted as soon as you press F5. So no need to go to project properties.

Hope this helps you?

How to fix crashes in release builds or how to enable debug information in release builds for VC projects?

September 25, 2008 2 comments

I will explain very briefly, how to fix crashes in release builds of VC applications.

Difference between debug and release builds

Well so what’s the difference between release and debug builds? The only major difference is that debug builds are for debugging purpose and release builds are for end users so for this purpose debug builds contain additional debug information, this is why there is considerable size differences in binaries for debug and release.

Common reasons for crashes in release builds

Well not too many steady reasons for crashes in release builds, but some to think of I will list here…

  1. Uninitialized variables
  2. Uninitialized pointers
  3. Always set a pointer to NULL after deletion
  4. Invalid iterators (std::iterator)
  5. Crap code
  6. Array index out of bounds for array and other stl classes.
  7. Too much optimization.
  8. RTTI project settings.

Check out above reasons if there is a crash in your release builds.

So tell us how to fix issues in release builds

  • Learn assembly. ūüėõ Joking! Naah not everyone has time to learn assembly but hey it’s¬†good to have basic knowledge on this topic.
  • Enable debug information for your release binaries. Ha, but will this increase the size of my release binaries. Nope! An extra file called a pdb (Program database) is created, which helps the JIT or any debugger to list out symbols used in your applications.
  • Always keep an updated set of symbols for all binaries; debuggers do download them automatically if you set _NT_SYMBOL_PATH to point to a symbol server. More information on how to set a symbol path is here.
  • Stop writing crap code.

Make my release build a debug build

If you are in the process of development and then you notice a crash while testing release build and you don’t know where the crash is taking place, do the following…

  1. Enable debugging for your release builds
  2. Go to C-C++/Optimizations/ and select Disable(Debug). If you don’t enable this bit you will see weird behavior while debugging. For e.g. addresses show as NULL even for valid objects on stack. By the way this might even be the cause of those crashes so initially you can leave it as it is but watch out for weird behaviors, don’t be surprised.
  3. Go to C-C++/Debug Info/ and select Program Database, this will generate a pdb file.
  4. If the output for a particular project is of .lib type then this much will suffice.
  5. Now for dlls and exes, follow steps 1 – 3.
  6. Go to Link tab/section and enable “Generate debug info”, this will add further debugging information to your exe or dll. Will show real deep stack traces with values of passed in parameters.

Yup now your release exe/dll/lib is now in full fledged debugging form. An additional thing to note is that _DEBUG is not enabled so AfxTrace, TRACE, TRACE0, TRACE1, TRACE2, ASSERTs, and VERIFYs won’t work.

Hope this helps you guys! ūüôā

Function return values live in VC watch window!

May 23, 2008 2 comments

Ever felt the need to watch over return¬†values from functions! Well I did always but unfortunately couldn’t come up with something! I was reading through one of John Robbins’ Bugslayer articles in MSDN and found an interesting tip related to seeing function parameters passed in by adding a watch statement over the ESP stack pointer variable.

So that got me thinking like “why not EAX”, huh that was easy! So this tip happens due to the tip given by¬†one of the¬†readers of John’s article in Bugslayer. I will put a link to that tip once I find it along with the author’s name.

Now let me tell you how the watch statement for our tip should look like…

  1. To watch string values returned by functions…
    (char*)EAX
  2. To watch integer values returned by functions…
    (int*)EAX
  3. To watch objects returned by functions…
    (CMyObject*)EAX

Now you may ask why do I need this, I can always use the auto variable¬†debug window. Well don’t forget that we are adding watch to¬†a register (EAX), so this means even raw function call’s return value is shown here.

Let me show you a live case where you may need this. See here I’ve got four function calls which does some strange operations but return values doensn’t matter hence they are not stored. It’s¬†just¬†useless¬†to store return values in temporary var just for the purpose of debugging, so above watches come handy.

DoOperationReturnBool(); // Returns bool, but returned values not stored
DoOperationReturnEnum(); // Cast EAX to appropriate enum type or int
DoOperationReturnCharPtr(); // Cast EAX to char pointer  or wide char ptr
DoOperationReturnObject(); // Cast EAX to appropriate object type, I haven’t tested this but should work

I am using this these days, yet to see a drawback, tell me if you find any! Have fun debugging ūüôā

Categories: Debugging, Visual studio Tags: , ,

Quick VC debugging tip!

So you have a dialog while you are debugging an application and you don’t know from where this dialog is popping up, huh, how many time has this happened!

So what normally people do is goto debug menu, press break and then goto threads and then double click on each thread to view call stack and then they find out. This is fine if you have a modal dialog, but what if it’s modeless?

So here is the solution, to break into an application while it’s running press “F12”. Yeah “F12”, can’t believe your eyes, do and believe!

So in order to debug into modeless dialogs you need to be real fast in pressing F12 ūüėČ . As soon as you start such a dialog press F12. This does the trick for me and must do for you too!

This same trick applies to any code that you are executing, at any time to break while debugging press “F12”. This doesn’t require visual studio to be running in the foreground. The foreground application should be the application that’s being debugged. After breaking into the process, continue as you would normally to find out where exactly and in which thread the execution is taking place.

%d bloggers like this: