First Steps with SubSonic

January 29, 2009

I have already written a couple of posts regarding code generation, which described how to generate C# or T-SQL code based on a database schema or table data.

My recent browsing expedition led me to T4 and SubSonic. After downloading and installing I found this introductory post on how to use SubSonic in your project.

These are the necessary steps (install-path is currently C:\Program Files\SubSonic\SubSonic 2.1 Final)

As you need to add a provider in the SubSonicService section, make sure the chosen name is not a C# keyword, as this will cause a compilation error.

  • Add a reference to the SubSonic.dll C:\install-path\SubSonic directory
  • Call SubSonic SubCommander from the command line:
C:\install-path\SubCommander\sonic.exe generate 
    /config \path-to-config /out \path-to-generated-source

This command creates a C# file for each table and view in your database, and wrappers for stored procedures.

Using this code, one can generate SQL statements disguised as C# code. I plan to leave my existing SQL code as it is, and write new functionality with the SubSonic framework.

I noticed that inserting records (via new Table() and .Save()) will overwrite the columns’ default values with NULL even if you do not set their values in code. You have to call the table object constructor with a true parameter to generate default values.

I still need to figure out the following questions:

  • How to run a stored procedure on a newly opened connection (connections are opened and closed automatically)
  • How to use SubSonic in conjunction with SqlDataSources for FormViews and GridViews

Generating a C# Stored Procedure Wrapper in TSQL

January 23, 2009

I love compiling languages. Whenever you change something that consequently does not fit the rest of the program, something breaks and you get a compiler error. Great!

If you develop database applications, this nice change-it-break-it mechanism stops to work, as the relation between database objects and their usage by a C# application is in no way hard-coded.

Therefore I developed a small piece of T-SQL code which will generate a static C# class which encapsulates all stored procedures in a given database in the form:

public static class MyStoredProc {
    public static SqlCommand MyProc1(SqlConnection conn)
    {
        SqlCommand cmd = new SqlCommand("MyProc1", conn);
        ...
        return cmd;
    }
    ...
}

The strategy is simple: loop through all desired procedures, and for each procedure loop through its parameters twice: once for the declaration, second time for the parameter values.

Create file and class header:

PRINT 'using System;
using System.Data;
using System.Data.SqlClient;
'

PRINT    'namespace ' + @NameSpace
PRINT    '{'
PRINT    'public static class ' + @Class + '
{'

Now we declare the loop for the procedure names (the code uses SQL2000 dictionary objects, which are also present on SQL2005; adaption should be simple):

DECLARE cP CURSOR FOR
SELECT  dbo.sysobjects.name
FROM     dbo.sysobjects
WHERE    (dbo.sysobjects.name NOT LIKE 'dt[_]%')
AND    ... other conditions ...
AND     (dbo.sysobjects.xtype = 'P')
ORDER BY dbo.sysobjects.name

OPEN cP
FETCH cP INTO @Proc

WHILE @@FETCH_STATUS = 0 BEGIN

PRINT '    public static SqlCommand ' + @Proc + '(SqlConnection conn'

Next we declare the cursor for the procedure parameters in the correct order:

DECLARE c CURSOR FOR
SELECT  dbo.syscolumns.name, dbo.systypes.name, dbo.syscolumns.length
FROM     dbo.sysobjects
INNER JOIN dbo.syscolumns ON dbo.sysobjects.id = dbo.syscolumns.id
INNER JOIN dbo.systypes ON dbo.syscolumns.xtype = dbo.systypes.xtype
WHERE    (dbo.sysobjects.name = @Proc) AND (dbo.systypes.name <> N'sysname')
ORDER BY dbo.sysobjects.name, dbo.syscolumns.colid

OPEN c
FETCH c INTO @Column, @Type, @Length

WHILE @@FETCH_STATUS = 0 BEGIN

For parameter declaration, we need to translate the SQL datatypes into C# datatypes. In this example, we use the following mapping:

int -> int?
datetime -> DateTime?
n/varchar -> string
bit -> bool?

PRINT  '        , ' +
    CASE @Type
    WHEN 'int' THEN 'int?'
    WHEN 'datetime' THEN 'DateTime?'
    WHEN 'nvarchar' THEN 'string'
    WHEN 'varchar' THEN 'string'
    WHEN 'bit' THEN 'bool?'
    ELSE '**datatype-not-supported**' END +
    ' ' + SUBSTRING(@Column, 2, LEN(@Column)-1)

All the wrapping method does is to create a SqlCommand using the SqlConnection:

PRINT ' )'
PRINT ' {'
PRINT '   SqlCommand cmd = new SqlCommand("' + @Proc + '", conn);'
PRINT '   cmd.CommandType = CommandType.StoredProcedure;'
PRINT '   cmd.CommandTimeout = 0;'

Then we loop through the parameters again to set the SqlCommand’s Parameters’ values. As shown above, parameter types are mapped to C# Nullable<T> types (question-mark notation), except for strings, as they can have the value null. Thus we have to treat string parameters differently from nullable (in C#) parameters:

OPEN c
FETCH c INTO @Column, @Type, @Length

WHILE @@FETCH_STATUS = 0 BEGIN

    IF @Type='nvarchar' OR @Type='varchar'
        PRINT ' cmd.Parameters.AddWithValue("' + @Column + '", ' +
            SUBSTRING(@Column, 2, LEN(@Column)-1) + ' != null ? (object)' +
            SUBSTRING(@Column, 2, LEN(@Column)-1) + ' : DBNull.Value);'
    ELSE
        PRINT ' cmd.Parameters.AddWithValue("' + @Column + '", ' +
            SUBSTRING(@Column, 2, LEN(@Column)-1) + '.HasValue ? (object)' +
            SUBSTRING(@Column, 2, LEN(@Column)-1) + '.Value : DBNull.Value);'

Finally, return the new SqlCommand:

PRINT '        return cmd;'
PRINT '    }'

Run this and paste the result into your C# application ;)


Listing Many-to-Many Relations in SQL Server 2005

January 22, 2009

SQL Server 2005 keeps its dictionary of database objects in system tables like sys.objects, sys.columns, sys.foreign_keys, and sys.foreign_key_columns (similar tables or views exist for SQL Server 2000, but in the dbo schema instead of sys).

Finding relations between tables is straight forward by querying sys.foreign_keys and joining with sys.objects.

To list many-to-many relations within a database, we have to first be aware that such a relation is implemented by 3 tables and 2 foreign keys. If the relation has attributes, then the table that represents the relation has more than two columns; if the relation is a typed relation, it has an additional foreign key constraint. So keep in mind that we can only query the implementation, but not the semantics of the implementation.

We can then define a set of constraints to rule out unlikely candidates for many-to-many relations: in my query, I chose the simplest constraint that the foreign key columns must not be nullable.

The following SELECT statement starts with a table, its columns, distinct foreign key constraints, and joins with two distinct (supposed) master tables:

SELECT Master1.name AS Master1, Master2.name AS Master2, 
    Detail.name AS Detail, DCol1.name AS Column1, DCol2.name AS Column2,
    ColumnCount.ColumnCount
FROM sys.objects AS Detail
INNER JOIN sys.columns AS DCol1 ON Detail.object_id = DCol1.object_id
INNER JOIN sys.foreign_key_columns AS FKC1 ON DCol1.column_id = FKC1.parent_column_id
    AND DCol1.object_id = FKC1.parent_object_id
INNER JOIN sys.objects AS Master1 ON FKC1.referenced_object_id = Master1.object_id
INNER JOIN sys.columns AS DCol2 ON Detail.object_id = DCol2.object_id
    AND DCol2.column_id <> DCol1.column_id
INNER JOIN sys.foreign_key_columns AS FKC2 ON DCol2.object_id = FKC2.parent_object_id
    AND DCol2.column_id = FKC2.parent_column_id
INNER JOIN sys.objects AS Master2 ON Master2.object_id = FKC2.referenced_object_id
INNER JOIN
    (SELECT object_id, COUNT(*) AS ColumnCount
     FROM sys.columns AS ColCount
     GROUP BY object_id) AS ColumnCount
    ON Detail.object_id = ColumnCount.object_id
WHERE (DCol1.is_nullable = 0) AND (DCol2.is_nullable = 0)
ORDER BY Master1, Detail, Master2

Automatically Generate MediaWiki Documentation of MS SQL Database Values

January 9, 2009

This is part 2 of the description of automating MediaWiki documentation using dbscript, and deals with documenting table data.

First, we need to specify which tables are to be documented: Go to the Project’s Value Scripts menu item, click New, select Script Type “XML”, enter an identifier for the script, and press Insert.

Next, go to the Tables item of the new Value Script. Select a table from the dropdown at the bottom of the page, and select Script Mode “Field Elements” (default).

I included the following tables of AdventureWorks in the documentation:

AddressType, AWBuildVersion, ContactType, CountryRegion, Culture, Currency, Location, SalesReason, SalesTerritory, ScrapReason, Shift, ShipMethod, UnitMeasure.

After you added the desired tables, they are displayed in the list Tables. Click each table name, and then its Columns menu.

From the Column dropdown, select the columns you want to add in your documentation, or press Insert All to include all columns. Columns can be deleted from the list by clicking Delete.

As the Value Script definition is complete, navigate to the Documentation Generator created in the previous post, and display its Parts page.

Under New Part, select Value Script, and click Create.

Enter a Sequence number, select an XSL (default “MediaWiki ValueScript DocGen”), select the newly created value script, and the database you imported the AdventureWorks schema from.

Specify an Alternate Page Section (e.g. “wikivalues”) to avoid overwriting the table definition in the default Page Section. Click Insert.

In the Documentation Generators menu click Generate, and generate the whole documentation, or just the newly created Part containing the data values.

Select the tables listed above to view data documentation samples on this project overview page. dbscript is available for download here.


Automatically Generate MediaWiki Documentation of MS SQL Database

January 7, 2009

I previously published a couple of posts on generating Wiki documentation in dbscript, and the current release 0.93 of dbscript puts all the steps together to automatically generate MediaWiki documentation of an MS SQL Server database schema, and values of selected tables.

This is an example of how to generate documentation for the AdventureWorks sample database. You need:

  • dbscript 0.93 or later installed
  • AdventureWorks installed
  • a MediaWiki installation with a user in the Bot group

In dbscript, create a Project AdventureWorks. Add a Database with dbo login credentials, and press Test Connection to mark the database connectable. (Alternatively, you can script the AdventureWorks database in Management Studio, and upload the generated sql file)

The Documentation Generator needs to handle tables differently from other database objects, as it needs to apply different XSL style sheets to the schema information.

Thus we create an Object Selection which selects every object except of tables: create an Object Selection “everything except tables”, and click the Objects page. Add Object Type “Table” with the option “Select included objects” selected. Add all other available object types with the “Select excluded objects” option.

Create a Documentation Generator, and select type “MediaWiki” (currently the only available option). Select the newly created Project Version. Enter the Wiki’s URL (http://mywiki.example.com), a section name (optional), the wiki bot’s username and its password.

If no section name is given, the whole page for each database object will be replaced. If you specify a section name, only this section is created or changed.

Next, go to the Parts page.

Under “New Part”, select “Project Version” and click Create. Enter a sequence number (e.g. 100), select an XSL (MediaWiki ProjectVersion overview), and a Page Title (“AdventureWorks Overview”) and save. This will create a summary page.

Go to the Parts page again, select New Part “Object”, click Create. Now we add tables to the documentation. Enter a sequence number (200), select an XSL (MediaWiki Table DocGen), and an Object Type “Table”.

Now we add all database objects other than tables: On the Parts page, select “Object”, click Create. Enter sequence number (300), select XSL (MediaWiki SourceCode DocGen), and specify Object Select “everything except tables”.

Go to the Generate page. Press the Generate button, and watch the progress log for error messages or progress updates.

The results of the documentation generator can be viewed here. dbscript is available for download here.


Page-specific actions in Selenium NUnit crawler

January 6, 2009

In my previous posts on Selenium and NUnit I described how to crawl your web application by following all the links on every page, and hashing the visited addresses.

My crawler can optionally reduce a page’s URL to a kind of signature consisting of the address and the names of its parameters. For example, the URL

http://myhost/app1/mypage.aspx?page=7&section=1

would be reduced to

mypage.aspx?page&section

If we want to add page-specific actions, the simplest approach is a huge switch/case statement which finds us the actions to be performed depending on the current address (and thus, depending on the signature of the current URL). Let’s define

delegate void PageTest();

and

string sUrlPattern = sUrl.Substring(sUrl.LastIndexOf("/") + 1);
if (sUrlPattern.Contains("?"))
{
    sUrlPattern = Regex.Replace(sUrlPattern, "=.+?&", "&");  // non-greedy
    sUrlPattern = Regex.Replace(sUrlPattern, "=.+", "");
}
List<PageTest> lifnTests = new List<PageTest>();

We can then add page tests to the list:

switch(sUrlPattern)
{
    case "mypage.aspx?page&section":
      lifnTests.Add(delegate() { TestMyPageWithPageAndSection(); });
      break;
    ...
}

The List<PageTest> now contains all the test functions that can be called because of the parameters of the current page.

foreach(PageTest pt in lifnTests)
{
    selenium.Open(sUrl);
    selenium.WaitForPageToLoad("60000");
    pt();
}

An action consists of the usual Selenium commands:

private void TestMyPageWithPageAndSection()
{
    selenium.Click("btnClickMe");
}

Of course, the necessary try/catch blocks, logging, etc need to be added to let NUnit run through an application test in case of an error.


Generating MediaWiki Documention of SQL Server Table Data

January 3, 2009

Recently I described how dbscript generates XML files from table data stored in MS SQL Server databases.

The upcoming version (0.93) of dbscript allows developers to generate wiki pages based on the contents of table data, which is useful to document tables containing “magic values”.

Among the XSL files that ship with dbscript is one that converts an XML value script into MediaWiki syntax.

This page on dbscript’s online help website contains a samples of a table used by dbscript.

dbscript is available for download here.


Follow

Get every new post delivered to your Inbox.