Building Visual Studio Solutions from Batch File

In my previous post, Building Visual Studio Solutions from the Command Line, I described the basic steps to be performed when one of my projects is to be built completely, or code needs to be generated.

This post gives a more detailed view of what is going on in the batch file. (Each batch file needs to be adjusted to the project it is used for in terms of functionality)

Handle batch parameters

Each function of the batch file can be invoked by a separate parameter, and a list of variables needs to keep track of whether the function is activated:

set dogenc=
... more variables

If no parameters are passed, show the help screen:

if not "%1"=="" goto params

echo.
echo parameters:
echo.
echo genc ... generate C#
... more info
goto end

Parse each command line parameter:

:params
if "%1"=="" goto endparams

if "%1"=="genc" (set dogenc=1& shift & goto params)
... more parameters
if "%1"=="all" (shift & set dogenc=1 & ... & set doall=1 & goto params)

echo unknown parameter "%1"
goto endend

:endparams

Add code to handle the various functions, such as checking, building, publishing, as described in the previous post.

Building Setup Projects

As msbuild does not build Setup Projects, we have to take a different approach:

"C:\path to\Microsoft Visual Studio 8\Common7\IDE\devenv"
    C:\path to\project.sln /build "Debug"
    /project C:\path to\setup.vdproj /projectconfig "Debug"

Generating Database Script

SQL Server 2000 provides a simple tool to create a SQL script with the definition of all database objects called scptxfr.

"C:\path to\Microsoft SQL Server\MSSQL\Upgrade\scptxfr"
    /s [server] /d [database] /p [password for sa] /f [filename] /a /h

Zip All

Finally, source code, executables, generated scripts and so on are zipped into version-specific archives using 7-zip, which can be controlled from the command line.

Summary

The batch file for my first project of this kind provides the following functionality:

• generate table triggers based on metamodel information
• generate C# constant definitions
• build executables (web, service)
• publish web applications
• build setup projects
• backup database (both as backup and as CREATE script)
• generate wiki information for developers
• generate wiki page stubs for online help
• create version-specific zip files of everything

Shrinking SQL Server Log Files

This script performs a truncate log operation followed by a shrink log file operation.

First it selects the current database name for the parameter of the BACKUP LOG command. Then the filename of the log file of the current database is retrieved to pass it to the SHRINKFILE command.

select * from sysfiles

declare @s nvarchar(100)

print 'truncating ' + db_name()
set @s = 'backup log ' + db_name() + ' with truncate_only'
exec (@s)

select @s = rtrim(name) from sysfiles where groupid = 0
print 'shrinking ' + @s
set @s = 'dbcc shrinkfile (' + @s + ', truncateonly)'
exec (@s)

select * from sysfiles

Recovering defective DVDs

I recently noticed that some DVD players refuse to play a DVD if part of a recording of it is unreadable (even though you wanted to watch a different recording on the same disc).

Time to check DVD quality: Nero CD-DVD Speed provides scanning and testing functionality, but does not help you in recovering partially damaged files.

I found this blog entry on recovering data files from defective discs, which lists a couple of solutions: Unstoppable Copier, CDCheck, and IsoBuster.

I tried IsoBuster, and as it comes with a zillion options, so I trusted the default values. Once it hit an unreadable sector, it asked whether to replace the sector with zeros or random data, and optionally applies your answer to any subsequent defective sector. I selected zeros, and hope that DVD players are clever enough to handle the zero parts.

Extended Functionality in graspx

To fine-tune the steps to automatically build a Visual Studio solution, I needed to implement some long-planned features: One of the targets in writing graspx was to extract all displayed text from the various controls on each ASP.Net form.

UI texts can be stored in a wide array of different controls and their attributes: label Text, hyperlink innerText, page Title, gridview EmptyDataText, validators ErrorMessage, and so on.

To make things more complicated, a control may not be found differently by its ID, but rather one needs to walk the form’s control hierarchy, if a control is placed within a FormView or GridView. Some texts may even be stored in a tag without an identifier, as is the case with ListItems in a DropDownList, or Columns within a GridView.

The new functionality in graspx covers these cases:

The LL command lists the value of all tag/attribute combinations listed in a separate parameter file, and thus equals the sequential execution of single L commands (uppercase “L” is used here for clarity).

The option -nc allows to define naming containers. If a control is found, the ID of the control is composed of the IDs of the parent controls. As an example from the setup files:

asp:GridView    .id
Columns         ix
asp:FormView    .id
ItemTemplate    tag
asp:Content     .ContentPlaceHolderID

For a GridView, the value of the ID attribute is used as naming ID, whereas for the Content (masterpage mechanism), it is ContentPlaceHolderID. Columns within a GridView are addressed by their index. A FormView may hold 3 different templates which are distinguished by their tag.

The working directory, which was the current directory in previous versions, can be set using the -d option; -r allows recursive searches through subdirectories.

The -nodyn option excludes all dynamic expressions ( < % # … % > ) from the search result.

The -utf8 option forces output in UTF8 encoding, which is not the default even for .Net console applications.

graspx is available for download here.

xmlns + SelectNodes = empty XmlNodeList

Trying to parse web.config files using SelectNodes, I found that I have two kinds of web.config files on my development PC, one with an xmlns declaration and one without.

<configuration>
<configuration
    xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0">

This should not really bother me, but it turns out that the SelectNodes method does not work as expected when an xmlns declaration is present. As this comment points out, a namespace manager definition is needed, and the namespace needs to be added to the manager. That’s because, the .Net runtime cannot do this for you.

The namespace of the XML document can be retrieved from DocumentElement.NamespaceURI.

As a ridiculous consequence, every XPath query has to include the namespace in every tag!

So, my previously single-line solution to iterate all user control namespaces in the web.config gained some weight, but fortunately it works:

XmlDocument docWC = new XmlDocument();
docWC.Load("web.config");

XmlNamespaceManager mgr =
    new XmlNamespaceManager(docWC.NameTable);
XmlNodeList xnl = null;
if (string.IsNullOrEmpty(docWC.DocumentElement.NamespaceURI))
{
    xnl = docWC.SelectNodes(
        "/configuration/system.web/pages/controls/add", mgr);
}
else
{
    mgr.AddNamespace("gr", docWC.DocumentElement.NamespaceURI);
    xnl = docWC.SelectNodes(
        "/gr:configuration/gr:system.web/gr:pages/gr:controls/"
        +"gr:add", mgr);
}

Extending default validity for self-generated SSL certificates

When I installed GForge recently, I had to generate an SSL certificate as part of the installation since GForge runs on https. However, the make-ssl-cert tool does not provide a way to customize the validity of the generated certificate, which is 30 days by default.

As it turns out, this problem is known for 3 years, and it is still not fixed in Ubuntu 8.04.

The easiest workaround is to edit the make-ssl-cert script using

whereis make-ssl-cert
sudo nano [path-to/]make-ssl-cert

and replace the line

openssl req -config $TMPFILE -new -x509 -nodes -out $output
    -keyout $output

with

openssl req -config $TMPFILE -new -x509 -days 365 -nodes
    -out $output -keyout $output

That’s it.

Installing GForge on Ubuntu 8.04 Hardy

I was looking for a program that lets me easily provide download and versioning functionality for the software that I create, and I decided to give GForge a try on my Ubuntu server. Installation turned out to be a bit messy, but finally I managed to make it work.

First, I tried the gforge package that came with Ubuntu using apt-get, but it failed to install the database component gforge-db-postgresql with the message

constraint "forum_group_forum_id_fk" for relation "forum" already exists

(see bug report). From this post I figured out that several constraint triggers with the same name were created in the SQL script, which had been removed according to the changelog.

After modifying the SQL script, installation raised another error with a malformed SQL statement containing

WHERE oldvalue=100 AND field_name='category_id'

(sorry I lost the original error message). Unable (and also unwilling) to debug unknown SQL code, I decided to give up.

Next, I tried the gforge package provided by Christian Bayle. I added the repository to my apt sources, and started installation. Unfortunately, the same SQL error in the WHERE clause caused the installation to terminate unsuccessfully.

I uninstalled everything using apt remove, and turned to my last choice:

Manual installation

From the download page, I chose gforge-4.5.19, because it seems to be the most recent stable version. The installation manual can be found on the Documentation page (User Documentation / Gforge Manual (PDF)). Even though the manual was written for 4.5.3, it is still accurate enough. Chapter 2.3 covers installation.

Since the earlier installation attempts already created a PostgreSQL database, I had to drop both the database and the user

sudo su - postgres
DROP DATABASE gforge;
DROP USER gforge;

The tcpip_socket option in postgresql.conf is not recognized anymore (as of version 8.3).

Setting up the Apache configuration was a bit of a mess, because the previous attempts had generated a gforge.conf (/etc/gforge/httpd.conf) file which contained instructions that Apache2 does not recognize. They also added references to modules that do not exist, such as auth_gforge_module.

Anyway, executing a manual installation from scratch should definitely work with fewer problems than I experienced.

Finally, I got the web application to run, but I had not enabled redirection which caused to redirect an HTTP request to HTTPS. Therefore, when I clicked on the link “Register as a site user”, which requires an SSL connection, the browser display the error message:

(site URL) has sent an incorrect or unexpected message
(error code -12263)

There were strong indications that the error was related to SSL.

Fine, I thought, just activate the https redirection, enable the VirtualHost section in the apache config file, create an SSL certificate, and then we’re done. So, I locate the instructions to set up an SSL server, which rely on a script called apache2-ssl-certificate which is not installed on Ubuntu Hardy. Well, actually it is missing from Ubuntu versions for one and a half years now.

Fortunately, the work-around is in the comments:

sudo make-ssl-cert /usr/share/ssl-cert/ssleay.cnf
    /etc/ssl/private/gforge.pem

In the VirtualHost section, add (or uncomment) the lines

SSLEngine On
SSLCertificateFile /etc/ssl/private/gforge.pem

In the end, I could successfully register a new user, added the Admin privilege via PostgreSQL, and now I am the proud and happy owner of a gforge installation

Let’s see what adventures await me there.

Building Visual Studio Solutions from the Command Line

Sometimes there are solutions that are technologically so simple that you don’t even think about them.

I develop a couple of projects, some of which are rather complex to build, as they require to generate code from database content.

Build tools can be complex and have a steep learning curve, Powershell might be an overkill since it’s useful if you handle .Net objects and containers, and I found it amazing what you can achieve with a little batch file.

This is a sketch of a project-specific build script:

First, declare log files and required tools. I found tee.pl to both display output on screen and write it to a log file.

set log=C:\path to build\build.log
set tee=perl C:\path to script\tee.pl
set osql=osql -S host -U user -P password -d database -n -w 1000

echo. > %log%

Next, check ASP.Net code for XHTML compliance and other criteria using graspx:

pushd \inetpub\wwwroot\path to web
(call ..\check2) | %tee% %log%
popd

One of my projects uses a metamodel to generate triggers, so generate a T-SQL script to drop and create triggers, and run that script using osql (SQL 2000) or sqlcmd (SQL 2005):

echo generating database stuff from metamodel | %tee% %log%
%osql% -Q "exec Generate_SQL_Code" -o sqlcode.sql
echo run generated code | %tee% %log%
%osql% -i sqlcode.sql | %tee% %log%

Next, generate C# const declarations from database content. (I find it’s a good idea to keep generated in a separate directory):

echo generating Consts.cs | %tee% %log%
%osql% -Q "exec dev_Generate_Consts" -o "C:\path to project\Generated\Consts.cs

The C# source code now matches the constants defined in the database. So we can now initialize the VS environment and build the project:

setlocal
call "C:\path to\Microsoft Visual Studio 8\VC\vcvarsall.bat" x86
pushd c:\temp
msbuild "C:\path to solution\project.sln"
popd
endlocal

If we build a web project, delete the PrecompileWeb directory that msbuild created:

rmdir /s /q "C:\path to\Visual Studio 2005\Projects\project\PrecompiledWeb"

Finally, update the online help wiki to include new aspx pages:

call createwiki.cmd

What have we got now?

• checked the source code
• generated database objects
• generated C# const definitions
• built VS projects

Next steps are backing up the development database and publishing the application.

Handling Errors in ASP.Net AJAX Requests

In my previous post I wrote about adding an exception handler in global.asax and adjusting web.config to display user-friendly error messages in ASP.Net applications.

While the strategy works fine, exceptions raised during an AJAX request are not handled by that solution. As I found out, you still get a Server Error page, for example if the SortExpression in a GridView within an UpdatePanel references an invalid column name.

To catch Ajax-generated exceptions, you need to add a AsyncPostBackError to the ToolkitScriptManager of your page or masterpage:

protected void ScriptManager1_AsyncPostBackError(object sender,
    AsyncPostBackErrorEventArgs e)
{
    Exception ex = e.Exception;
    Session["Application_Error"] = ex;
    Session["Application_Request"] = Request;
    Response.Redirect("error.aspx");
}

This sample code uses the same Session variables as in the previous post, and explicitly redirects to the same error page.

Scheduling Backups on SQLServer Express

Management Studio Express does not provide a user interface to schedule backup jobs, but you can easily find a couple of solutions on the web, such as here, here, and here.

The backup solution I came up with was one .sql file each for full and incremental backup, and two .cmd files which execute the respective .sql files via sqlcmd.

The full backup SQL file looks like this:

PRINT 'backup.sql ' + CONVERT(VARCHAR, GETDATE(), 120)

DECLARE @backupSetId AS INT
DECLARE @Filename NVARCHAR(256)
DECLARE @Database NVARCHAR(256)
DECLARE @Backup NVARCHAR(256)

SET @Database = N'<name of database>'
SET @Filename = N'E:\Backups\<name of database>.' +
    REPLACE(CONVERT(NVARCHAR, GETDATE(), 102), '.', '-') +
    N'.bak'
SET @Backup = @Database + N' Full Database Backup'

BACKUP DATABASE @Database
TO DISK = @Filename
WITH NOFORMAT, NOINIT,
NAME = @Backup,
SKIP, NOREWIND, NOUNLOAD, STATS = 10

SELECT    @backupSetId = position
FROM    msdb..backupset
WHERE    database_name = @Database
AND        backup_set_id =
    (SELECT MAX(backup_set_id) FROM msdb..backupset
        WHERE database_name=@Database )

IF @backupSetId IS NULL
    PRINT N'Verify failed. Backup information for database ''' +
        @Database + N''' not found.'
ELSE
    RESTORE VERIFYONLY FROM  DISK = @Filename
    WITH FILE = @backupSetId, NOUNLOAD, NOREWIND

This piece of code is repeated for every database you want to backup.

The incremental backup file has the WITH DIFFERENTIAL option in the BACKUP command, and excludes verification.

The batch file which executes the backup.sql file simply calls sqlcmd:

sqlcmd -S <your db server name>
    -i <path to>\backup.sql
    -o <path to>\backup.log

Next, use the Scheduled Tasks control panel application to schedule the full backup once a week, and the incremental backups daily: Use a weekly schedule and select the days of week to define when the backups should run.