How to debug a DLL or EXE

This article shows a technique to attach the Visual Studio debugger to a running process in the case of a user defined surface (DLL) and in the case of a user defined operand called UDOC (EXE). The technique is shown here using Microsoft Visual Studio Express 2013 Desktop.

Compile the DLL or EXE
Open OpticStudio and select the user defined surface or operand
Attach to Process
Set Breakpoints
Run the DLL/EXE
Stop the debugger
Distributing the DLL/EXE in Release Mode

Michael Humphreys, Sandrine Auriol and Tim Gustafson
Programming Zemax

Description of the steps

We are starting from C++ or C# projects in Visual Studio that create a user defined surface (C++) and a user defined operand (C++ or C#).
The steps to create those user defined features are not detailed here.  

Compile the DLL or EXE in Debug Mode

  •  First method: First we need to compile the DLL or EXE in Debug Mode. For example, below we are compiling a user defined surface (DLL).

Debug mode
Figure 1: Debug mode

Debug output
Figure 2: Debug Output

Once compiled, we need to copy the DLL and PDB file from the Debug folder of our project to C:\Program Files\Zemax OpticStudio\DLL\Surfaces.
Normally (when using the DLL in release mode) we only copy the DLL file. But the PDB file (Program database) is needed here because it stores the debugging information about programs.
 Copy dll and pdb
Figure 3: Copy DLL and PDB

However, because the DLL is in the default build folder for the solution and not the required OpticStudio folder, we will have to manually copy over the DLL and PDB files every time we modify anything in the DLL and recompile.

  • 2nd method: change the output folder of the DLL or EXE.
Another solution is to change the output folder of the DLL or EXE to “C:\Program Files\Zemax OpticStudio\DLL\....” before compiling. Note that in order to successfully save the DLL or EXE in that folder, we need to run Visual Studio with elevated privileges as an administrator.   
Visual Studio as Administrator
Figure 4: Run Visual Studio as Administrator

Once we have our solution opened with elevated privileges, we can change the build output directory by right clicking on the Project Properties, selecting Properties, and then change the Output Directory under the General tab.
  • In C++:

Project properties
Project properties
Figures 5 and 6: Project properties
  • In C#: this can be changed under Project Preferences > Build > Output directory.

Project properties output path
Figure 7: Output path

Once the output folder has been changed, we can compile. All build files, including the DLL/EXE and PDB will be in the proper folder for OpticStudio to load.

Open OpticStudio and select the user defined surface or operand

The DLL or EXE is now ready to be used and we have opened OpticStudio.
  • To use the user defined surface DLL, we will add it to an optical system; for out example, we will use a singlet. The surface 1 is now a user defined surface that is using the DLL in Debug mode:

OpticStudio user defined surface
Figure 8: User-defined surface in OpticStudio

  • To use the user defined operand UDOC which is an EXE, we can load the UDOC into the Merit Function.  Make sure to set the Timeout (ms) to something extremely high so we can actually debug the UDOC in Visual Studio without having OpticStudio throw a timeout error.

Figure 9: UDOC in OpticStudio

At this point do not update the system or the Merit Function yet. An update will cause the user defined surface or UDOC to execute. If there are errors in the code, it may cause a crash before we can set a breakpoint.

Attach to Process

Back to Visual Studio, we are now ready to start debugging.
  • If the user extension is a DLL, we will attach the debugger to the OpticStudio.exe process.
  • If the user extension is an EXE, we will attach the debugger to our application process.
Let’s see how to do that.
  • DLL Method – Attach to OpticStudio

We can attach the debugger to OpticStudio running process :

Attach to process
Attach to process
Figures 10 and 11: Attach to Process

Some comments about the settings:
  • The “Attach to” box will select the type of code to be debugged.
    • If we are using C++ we will have to use Native code
    • If we are using C# we will have to use Managed code
  •  In “Available Processes”, make sure to select the appropriate OpticStudio process. It may be easier to have only one OpticStudio instance open.  
  • EXE Method – Attach Directly To UDOC

To debug a UDOC, we will attach the UDOC application itself rather than attaching it to OpticStudio.  Therefore, we will need to set a “pause” in the UDOC code itself (either a Console.ReadKey() for C# or system(“pause”) for C++) to pause the actual execution. 
In C#, the code might look like:

Attach to udoc
Figure 12: Attach to UDOC

Now, when we update the Merit Function, we will see a Command Line Window open and the message Attach debugger (Ctrl+Alt+P) then press any key… will be displayed. 

Update merit function
Figure 13: Update Merit Function

Now, we want to attach the UDOC itself to the debugger.  Make sure to have the correct code selected for the UDOC. Below, we have selected “Managed code” as the code is in C#.

Udoc debugger
Figure 14: UDOC debugger

Note that with a UDOC, we will have to attach a process every time we update the Merit Function and want to inspect the UDOC.

Set Breakpoints

Then we can set a few breakpoints inside the DLL code to investigate the values being passed.
For example, we’ve set a breakpoint on line 42 for the user defined surface.
Figure 15: Breakpoint

Run the DLL/EXE

In OpticStudio, we can click on the surface and it starts the debugger:

Run dll
Figure 16: Run DLL

For running the UDOC, we will simply update the Merit Function.
In Visual Studio, the yellow arrow on the breakpoint shows the code to be executed next.
For example, let’s look at the variables when debugging the user defined surface:

Debugging dll
Figure 17: Debugging DLL

The window DEBUG > WINDOWS > LOCALS will display the values being passed between the DLL and OpticStudio. In that window, we will find the two types of data structures passed between OpticStudio and DLL: User Data (UD) and Fixed Data (FD):
  • UD contains the rays x, y, z locations on a plane tangent to the surface vertex (the DLL will compute and return them for the real surface), the ray direction cosines l, m, n and the surface normals ln, mn, nn (which are computed  and returned) and other data like index, transmission, path length.

Figure 18: UD
  • FD mainly contains data the DLL cannot change: the surface data in the Lens Data Editor (radius, conic, parameters, etc.), the ray wavelength, polarization state and Type and Numb parameters indicating what data OpticStudio wants from a given call to the DLL.

Figure 19: FD

For example, here we can see the index of refraction of N_BK7 being passed from OpticStudio to the DLL.

Stop the debugger

To end the debugging just click on the red Square to stop debugging.

Stop debugger
Figure 20: Stop debugger

Distributing the DLL/EXE in Release Mode

When we have finished debugging the DLL/EXE, we need to make sure to change the configuration from Debug back to Release and recompile the solution.  The libraries for Debug configurations are not redistributable while the libraries for Release configurations are redistributable.


The technique to debug a user defined feature was demonstrated in the case of a DLL (user defined surface) and an EXE (user defined operand). It can be applied for any user defined capabilities that OpticStudio has: DLL, user defined operands, ZOS-API extensions and user-analysis.
Other useful resources:
Attach to Running Processes with the Visual Studio Debugger:
Navigating through Code with the Debugger:
Update for Visual C++ 2013 and Visual C++ Redistributable PackageUpdate for Visual C++ 2013 and Visual C++ Redistributable Package: