Programmatically Add External Tool to Visual Studio

I tried to find some information on adding an External Tool to Visual Studio using a Setup Project, and came across this Social.MSDN entry.

While it deals with Visual Studio 2005, the information can easily be modified to work in later versions of Visual Studio.

In your Setup Project’s Registry View, add the following keys:

For Visual Studio 2005

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\8.0\External Tools\

For Visual Studio 2008

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\9.0\External Tools\

For Visual Studio 2010

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\10.0\External Tools\

Below each External Tools key, add a key identifying your tool.

Under the key for your tool, you need to add ALL values (from MSDN above):

“ToolArg”=”$(ItemPath)”
“ToolOpt”=dword:00000008
“ToolCmd”=”notepad.exe”
“ToolDir”=”$(TargetDir)”

The flags that can be or’ed together in “ToolOpt” are the following:

#define TOOLOPT_ISAPPGUI      0x01
#define TOOLOPT_CLOSEONEXIT   0x02
#define TOOLOPT_PROMPTFORARGS 0x04
#define TOOLOPT_USEOUTPUTWIN  0x08
#define TOOLOPT_SAVEALLDOCS   0x10
#define TOOLOPT_USETASKLIST   0x20
#define TOOLOPT_UNICODE       0x40

Note that Setup Project expects decimal notation for DWORD values.

At first, my tool was not displayed because I did not provide the ToolDir value as it is not relevant for execution.

Providing all the values displays the tool’s menu as disabled (grayed).

Opening the External Tools dialog and clicking OK enabled to entry for my new external tool.

VS 2010 Setup Project: Error updating dependencies

I successfully created my Web Setup Project, but a couple of days later the setup project stopped compiling and raised errors instead:

ERROR: Unable to update the dependencies of the project. The dependencies for the object ‘MyAssembly.DLL’ cannot be determined.

Clicking the Refresh Dependencies command in the context menu of the setup project’s Dependencies raised the error

The operation could not be completed.

These errors occur in VS 2010, and seem to be around for several years now (connect, connect, connect).

Workaround

From the workarounds section of this Connect entry, this solution worked for me:

1. Open .VDPROJ file (in a text editor)
2. Find the "Hierarchy" section.
   Delete everything so the section looks like this:

    "Hierarchy"
    {
        
    }

3. Find the "File" section.
   Delete everything so the section looks like this:

        "File"
        {
            
        }

4. Reload the project
5. Rebuild the project.

I removed the whole sections “Hierarchy” and “File”, which also worked.

However, the whole procedure seems cumbersome considering you may have more than one setup project in a solution, and probably more than one solution you are developing or maintaining.

Fix

The error still seems to be present in VS 2010 SP 1. Fortunately, MS in the meantime provides the hotfix KB2286556 which solved the problem for now.

Pitfalls installing Web Setup .msi on IIS 7

I created a Web Setup project in Visual Studio 2010 for a .Net 4 web application to install on a Windows 2008 Server running IIS 7. The prerequisite knowledge is described in my previous posts.

Upon starting the .msi file (or the accompanying setup.exe), I received the error message

“The installer was interrupted before Application could be installed. You need to restart the installer to try again.

Click “Close” to exit.”

This Social MSDN entry brought the first step to the solution: activate logging using the command line

msiexec /i <path to msi file> /l*vx C:\SomeDirectory\SomeFilename.log

I needed to register the .Net 4 framework in IIS using the commands

cd C:\Windows\Microsoft.NET\Framework\v4.0.30319
aspnet_regiis.exe -i
aspnet_regiis.exe -ga MyDomain\MyUserName

After registering the framework version, the next error showed in these lines in the log file:

INFO   : [SetTARGETSITE]: Custom Action is starting...
INFO   : [SetTARGETSITE]: CoInitializeEx - COM initialization Apartment Threaded...
ERROR  : [SetTARGETSITE]: FAILED: -2147221164
ERROR  : [SetTARGETSITE]: Custom Action failed with code: '340'
INFO   : [SetTARGETSITE]: Custom Action completed with return code: '340'

The cause for these errors was that the IIS 6 Management Compatibility feature was not installed on the server.

Installation of this feature is easy: go to Control Panel\Programs and Features\”Turn Windows features on or off” and check the IIS 6 Management Compatibility checkbox. (on Server 2008, this can be done in the Server Manager, under Roles, Webserver (IIS)).

Visual Studio requires IIS 6 Management Compatibility to be installed on the developer machine:

To enable Visual Studio to create and use local IIS Web sites, you must enable metabase compatibility. This lets Visual Studio interact with the IIS metabase and with the IIS 7.0 configuration store.

I also found a similar statement in the IIS forum:

Visual Studio 2010 web setup project still only targets older IIS versions. This is why the compatibility features are required.

So how can a developer detect if the IIS 6 Compatibility feature is installed?

The registry path HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\InetStp contains the IIS version in the keys MajorVersion and MinorVersion.

This page contains the list of IIS components that can be detected under the registry path HKEY_LOCAL_MACHINE\Software\Microsoft\InetStp\Components. (see also SO, CodeProject), and the relevant keys seem to be Metabase and ADSICompatibility (both 1 of the compatibility feature is installed).

This page on Metabase compatibility explains the architecture of this feature:

It supports the Admin Base Objects (ABO) interface, also known as IMSAdminBase, as well as the ADSI and WMI providers that were built on top of ABO in IIS 6.0.

This component is not installed by default because IIS 7.0 itself is not using it.

My understanding is that the DirectoryEntry class which queries ADSI can only work if ABO is installed. ABO itself is a COM library, so the required classes and interfaces must be stored in the registry.

The file Iadmw.h of the IIS SDK contains the definition

DEFINE_GUID(IID_IMSAdminBase_W, 0x70b51430, 0xb6ca, 0x11d0, 0xb9, 0xb9, 0x0, 0xa0, 0xc9, 0x22, 0xe7, 0x50);

and indeed, the key HKEY_CLASSES_ROOT\Interface\{70B51430-B6CA-11D0-B9B9-00A0C922E750} has the default value IADMCOM if the Metabase compatibility is installed.

Installing additional Virtual Directories in a Web Setup Project

A Visual Studio Web Setup Project (or Deployment Project) lets you define the contents of a Web Application Folder by adding Content Files and Primary Output of a Web project in the File System view.

Additionally, using the menu command Add Special Folder/Web Custom Folder, one can add more web directories to the setup project, which will end up in the file system under c:\inetpub\wwwroot and will define a separate IIS application for each folder.

In one of my projects I needed to add additional Virtual Directories located at web root and pointing to subdirectories of the web app located in the Web Application Folder. (some URLs generated by the app were root-based, and will be cleaned up, so please don’t ask)

This can accomplished by creating an installer class derived from System.Configuration.Install.Installer, and using one of the many APIs available to administrate IIS (MSDN on IIS 6, MSDN on IIS 7). The project containing the installer class must be added to the setup project’s File System view.

I chose to use the MSDN sample to create Virtual Directories using System.DirectoryServices.DirectoryEntry as described in my previous post. The installer class is called by the web installer using the Custom Actions mechanism.

The find the physical location of the installed directories, you can either derive the path information from Assembly.GetExecutingAssembly().FullName or by passing the installation directory as a parameter to the Custom Action.

  • Custom Actions
  • Install
  • Primary output from MyWebInstaller
  • CustomActionData /targetdir=[TARGETDIR]

You can name the parameter as you like, but the [TARGETDIR] is a pre-defined web installer property.

The parameter is accessed via the Context.Parameters collection in the installer:

var targetdir = Context.Parameters["targetdir"];

This value is used to create the virtual directory in web root:

const string WebRoot = "IIS://127.0.0.1/W3SVC/1/Root";
CreateVDir(WebRoot, "MyRootDir", targetdir.Replace(@"\\", @"\") + "MyDir");

Notes:

  • This post has a complete walk-through on web setup projects and custom actions.

Adding Application Directory to PATH variable in Visual Studio Setup Projects

To add the directory where a program is installed to the PATH environment variable requires a Custom Action in your Visual Studio Setup project.

Create a library (.dll), follow the instructions in this article and paste the C# code into a class. (VB.Net version is here)

This code adds the program’s directory to the PATH variable stored in HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Environment.

However, to notify the system that the PATH variable has changed, a broadcast message needs to be sent to all windows using the Windows function SendMessageTimeout. This function can be declared using the DllImport attribute:

[DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool SendMessageTimeout(
  IntPtr hWnd,
  int Msg,
  int wParam,
  string lParam,
  int fuFlags,
  int uTimeout,
  out int lpdwResult
);

public const int HWND_BROADCAST = 0xffff;
public const int WM_SETTINGCHANGE = 0x001A;
public const int SMTO_NORMAL = 0x0000;
public const int SMTO_BLOCK = 0x0001;
public const int SMTO_ABORTIFHUNG = 0x0002;
public const int SMTO_NOTIMEOUTIFNOTHUNG = 0x0008

Call this function from a .Net method passing the correct parameters after manipulating the registry:

static void BroadcastEnvironment()
{
  int result;
  SendMessageTimeout((IntPtr)HWND_BROADCAST, WM_SETTINGCHANGE, 0, "Environment",
    SMTO_BLOCK | SMTO_ABORTIFHUNG | SMTO_NOTIMEOUTIFNOTHUNG, 5000, out result);
}

After running the installer, open the cmd prompt, and type PATH. You will find your directory added to the system path.

Other possibilities to implement this functionality are to edit the created .msi file, or to add the registry key of the Environment variable to the setup project. However, this seems to be a destructive method during uninstall.

vdproj: Add Application to Context Menu of Existing File Type in VS Setup Project

The registry associates file extensions with file types and their applications.

Taking Visual Studio as an example, the key HKCR\.sln points to the file type VisualStudio.Launcher.sln (HKCR being the abbreviation for HKEY_CLASSES_ROOT).

Under HKCR\VisualStudio.Launcher.sln\Shell, the registry lists all commands that are available from Explorer’s context menu on a .sln file.

I investigated how to add my application to this key using a Visual Studio Setup Project (.vdproj), especially, how to retrieve the installation directory selected by the user during setup.

In the Registry view of the Setup Project

  • locate the HKEY_CLASSES_ROOT key.
  • select New Key and add the file type, e.g. VisualStudio.Launcher.sln.
  • select New Key, and add “Shell”
  • select New Key, and add either “[ProductName]” or the text you want to display in the context menu
  • select New Key, and add “Command”
  • in the right-hand pane, Select New/String Value, and delete the name of the key. The name will now be displayed as “(Default)”.
  • select the default key, go to the Properties window, and set the value to “[TARGETDIR]myprogram.exe” “%1”.

(The double quotes have to be typed in the property value to handle spaces in executable and document paths).

Upon installation, your program will now appear in the context menu of the files associated with this type.

The difficulty for me was to find the macros that can be used in a setup project. That’s because Microsoft does not refer to them as installer macros or installer variables, but rather properties.

Here is the complete list of Setup Project Macros.

Building Visual Studio Solutions from Batch File

In my previous post, Building Visual Studio Solutions from the Command Line, I described the basic steps to be performed when one of my projects is to be built completely, or code needs to be generated.

This post gives a more detailed view of what is going on in the batch file. (Each batch file needs to be adjusted to the project it is used for in terms of functionality)

Handle batch parameters

Each function of the batch file can be invoked by a separate parameter, and a list of variables needs to keep track of whether the function is activated:

set dogenc=
... more variables

If no parameters are passed, show the help screen:

if not "%1"=="" goto params

echo.
echo parameters:
echo.
echo genc ... generate C#
... more info
goto end

Parse each command line parameter:

:params
if "%1"=="" goto endparams

if "%1"=="genc" (set dogenc=1& shift & goto params)
... more parameters
if "%1"=="all" (shift & set dogenc=1 & ... & set doall=1 & goto params)

echo unknown parameter "%1"
goto endend

:endparams

Add code to handle the various functions, such as checking, building, publishing, as described in the previous post.

Building Setup Projects

As msbuild does not build Setup Projects, we have to take a different approach:

"C:\path to\Microsoft Visual Studio 8\Common7\IDE\devenv"
    C:\path to\project.sln /build "Debug"
    /project C:\path to\setup.vdproj /projectconfig "Debug"

Generating Database Script

SQL Server 2000 provides a simple tool to create a SQL script with the definition of all database objects called scptxfr.

"C:\path to\Microsoft SQL Server\MSSQL\Upgrade\scptxfr"
    /s [server] /d [database] /p [password for sa] /f [filename] /a /h

Zip All

Finally, source code, executables, generated scripts and so on are zipped into version-specific archives using 7-zip, which can be controlled from the command line.

Summary

The batch file for my first project of this kind provides the following functionality:

  • generate table triggers based on metamodel information
  • generate C# constant definitions
  • build executables (web, service)
  • publish web applications
  • build setup projects
  • backup database (both as backup and as CREATE script)
  • generate wiki information for developers
  • generate wiki page stubs for online help
  • create version-specific zip files of everything