Handling the “Size Limit” in AD Queries

February 10, 2017

I created a small tool to mirror AD data into an SQL Server database. The AD queries essentially looked like this

var conn = new ADODB.Connection();
conn.Open("Provider=ADsDSOObject", "", "", 0);

const string where = "objectCategory='group' ";      

var qry = string.Format(@"SELECT objectCategory, displayName, [more attributes]
FROM 'LDAP://{0}/{1}' 
WHERE {2}", server, start, where);

object recs;
var rs = conn.Execute(qry, out recs, 0);

for (; !rs.EOF; rs.MoveNext())
{
    // ... process record
}

The attributes available in AD have been taken from here.

The code worked fine for many months, until one day it threw an exception:

System.Runtime.InteropServices.COMException (0x80072023): The size limit for this request was exceeded.
at ADODB.RecordsetClass.MoveNext()

It turned out that the query result suddenly exceeded 3000 records, which may or may not be a magic or configurable limit of fetches for a single AD query – probably also including the number of records each fetch returns. Who knows.

Thanks goes to the internetz, which provided me with a solution which now fetches more than 3000 records. Just replace the conn.Execute() part with

var cmd = new ADODB.Command {ActiveConnection = conn, CommandText = qry};
cmd.Properties["Page Size"].Value = 10000;
cmd.Properties["Timeout"].Value = 30;
cmd.Properties["Cache Results"].Value = false;

Fixing PDFSharp hangs

January 12, 2017

To analyse a couple of PDF files whether they contain only images, I used the latest release build of PDFsharp, version 1.32.

However, when processing a certain file (of unknown origin) using code found in an SO answer

public static IEnumerable ExtractText(this PdfPage page)
{       
    var content = ContentReader.ReadContent(page);      
    var text = content.ExtractText();
    return text;
}   

the ExtractText() function simply would not return.

I upgraded to the most current build 1.50 beta 3, included the source in my project, and ran it in Debug mode, where execution halted in the file PDFsharp\src\PdfSharp\Pdf.Content\CParser.cs line 163 failing an assertion:

#if DEBUG
    default:
        Debug.Assert(false);
        break;
#endif

Without digging too deep into the analysis of PDF files, it was clear that the PDF contained a CSymbol that is not being handled by the library, and thus (most likely) ended up in an infinite loop inside CParser.ParseObject().

I fixed this by replacing the Debug.Assert statement with

        throw new Exception("unhandled PDF symbol " + symbol);

which fixed the situation for me.


Unit Testing ASP.Net MVC 5 Controllers using Rhino Mocks

February 12, 2016

Unit Testing ASP.Net MVC Controllers is not as straight-forward as you may thing, because as you get started, you face questions such as: how to create an HttpContext, an HttpContextBase, should you prefer mocking, and which mocking framework, etc.

After a day of googling and tinkering, I finally came up with a working solution using Rhino.Mocks and MvcContrib, both downloaded via nuget. To analyze Owin black magic, I used the source code repositories of Katana on Codeplex and symbolsource.

[TestClass()]
public class HomeIndexControllerTests
{
  [TestMethod()]
  public void HomeIndexTest()
  {

The authentication routine uses async calls, such as FindByNameAsync(). I tried to declare the test method as public async Task, but after starting the test, the IDE claimed it’s busy, but did not execute the test.

Without Task.Run(), the async method would fail throwing an AggregateException.

I found the work-around to encapsulate the whole test inside a Task.Run():

    Task.Run(() =>
    {
      var builder = new TestControllerBuilder();

First, we mock the ApplicationUserManager

      var us = MockRepository.GenerateStub<IUserStore<AppUser>>();
      var aum = MockRepository.GenerateStub<ApplicationUserManager>(us);

My class for storing users in the user database is called AppUser. The variable appuser is set to null for unauthenticated requests, and contains a valid object if authenticated:

      AppUser appuser = null;
      //appuser = new AppUser { .... values .... };  // uncomment for authenticated request

These stubs return the defined user object:

      us.Stub(u => u.FindByNameAsync(""))
        .IgnoreArguments()
        .Return(Task.FromResult(appuser));
      aum.Stub(m => m.FindByNameAsync((ClaimsIdentity)null, ""))
        .IgnoreArguments()
        .Return(Task.FromResult(appuser));

Create the Owin context and register the user manager:

      var owin = new OwinContext();
      owin.Set(aum);

Use the builder to create the controller under test:

      var controller = builder.CreateController<C.Home.IndexController>();

To test unauthenticated requests, we check whether the class or the method have an [Authorize] attribute

      if (appuser == null)
      {
        var type = controller.GetType();
        var attributes = type.GetCustomAttributes(typeof(AuthorizeAttribute), true);
        var methodInfo = type.GetMethod("Execute" 
          /*, new Type[] { ....parameter types.... } */
        );
        var methodAttributes = methodInfo.GetCustomAttributes(typeof(AuthorizeAttribute), true);
        Assert.IsFalse(attributes.Any() || methodAttributes.Any(), "Unauthorized request not allowed");
      }

Next, we wire up the HttpContext and Owin, and set the request’s cookies:

      var context = controller.HttpContext; 
      // the context is already mocked, so we can stub() it

      var dict = new Dictionary<object, object>();
      dict.Add("owin.Environment", owin.Environment);
      context.Stub(c => c.Items).Return(dict);
      var request = controller.Request;
      request.Stub(r => r.Cookies).Return(new HttpCookieCollection());

Next, we create a principal from the user object (see here for setting the name claim)

      var user = new GenericPrincipal(
        new ClaimsIdentity(
          new Claim[] { 
            new Claim(ClaimTypes.Name, appuser == null ? "" : appuser.UserName) }),
        new string[0]);
      context.User = user;

Then we create a fake HttpContext which matches the mocked HttpContextBase object created above:

      HttpContext.Current = new HttpContext(
        new HttpRequest("", "http://tempuri.org", ""),
        new HttpResponse(new StringWriter())
      );
      HttpContext.Current.Items["owin.Environment"] = owin.Environment;
      if (appuser != null)
        HttpContext.Current.User = user;

We are now ready to execute the controller method

      builder.InitializeController(controller);
      var result = controller.Execute();

And check the results: if the request is successful, the result is typically a ViewResult.

      Assert.IsInstanceOfType(result, typeof(RedirectToRouteResult));
      Assert.IsInstanceOfType(result, typeof(ViewResult));

We can now test the result values, and end the Task definition:

    }).GetAwaiter().GetResult();
  }
}

 


NHibernate SELECT MAX()

February 3, 2016

A form used to edit or add records was to set a number field to the highest assigned integer value plus one.

This is typically achieved by writing

SELECT Max(NumberField)+1 FROM [Table]

and in NHibernate you write something like

result.Number = session.Query<Table>()
    .Max(t => t.Number) + 1;

where Number is defined as int.

While this solution is principally correct, it fails if the table has no records:

Server Error in '/' Application.
Value cannot be null.
Parameter name: item
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.ArgumentNullException: Value cannot be null.
Parameter name: item
Stack Trace:
[ArgumentNullException: Value cannot be null.
Parameter name: item]
 System.ThrowHelper.IfNullAndNullsAreIllegalThenThrow(Object value, ExceptionArgument argName) +4195637
 System.Collections.Generic.List`1.System.Collections.IList.Add(Object item) +32
 NHibernate.Util.<>c__DisplayClass4.<AddAll>b__2() +13
 NHibernate.Util.ArrayHelper.AddAll(IList to, IList from) +445
 NHibernate.Engine.Query.HQLQueryPlan.PerformList(QueryParameters queryParameters, ISessionImplementor session, IList results) +573
 NHibernate.Impl.StatelessSessionImpl.List(IQueryExpression queryExpression, QueryParameters queryParameters, IList results) +329
[GenericADOException: Could not execute query[SQL: SQL not available]]
 NHibernate.Impl.StatelessSessionImpl.List(IQueryExpression queryExpression, QueryParameters queryParameters, IList results) +379
 NHibernate.Impl.AbstractSessionImpl.List(IQueryExpression queryExpression, QueryParameters parameters) +145
 NHibernate.Impl.AbstractQueryImpl2.List() +117
 NHibernate.Linq.DefaultQueryProvider.ExecuteQuery(NhLinqExpression nhLinqExpression, IQuery query, NhLinqExpression nhQuery) +36
 NHibernate.Linq.DefaultQueryProvider.Execute(Expression expression) +50
 NHibernate.Linq.DefaultQueryProvider.Execute(Expression expression) +11
 System.Linq.Queryable.Max(IQueryable`1 source, Expression`1 selector) +283

The problem is, of course, that SELECT MAX(NumberField) on an empty table results in a single record containing a NULL value for the computed column, and the NHibernate mapper throws an exception trying to assign the NULL value to the int property.

Additionally, the SELECT statement above should also be written as

SELECT ISNULL(MAX(Number), 0) + 1

to generate 1 for the first record, which already points in the right direction.

The correct NHibernate statement is therefore

result.Number = session.Query<Table>()
    .Max(t => (int?) t.Number) ?? 0 + 1;

casting the C# int property to a Nullable<int>, which accepts the NULL value resulting from MAX(). The ?? operator is the equivalent of T-SQL’s ISNULL() function.

Fundamentally, the behavior is caused be the definition of the Queryable.Max() method:

public static TResult Max<TSource, TResult>(
  this IQueryable<TSource> source, 
  Expression<Func<TSource, TResult>> selector);

If the selector returns int, then the method also returns int. Could the result type not have been declare as Nullable<TResult>?

What does Linq to Objects do? This simple code

IEnumerable<int> values = new int[0];
Console.WriteLine(values.Max());

throws an InvalidOperationException with the message

Sequence contains no elements

as the corresponding Max() method also does not handle Null values. The correct way in Linq to Objects is to call DefaultIfEmpty() to replace an empty enumerable with one containing a single-element default value:

IEnumerable<int> values = new int[0];
values = values.DefaultIfEmpty();
Console.WriteLine(values.Max());

So, I experimented a bit to find a meaningful solution

public static T? max<T>(IEnumerable<T> values) 
  where T: struct, IComparable<T>
{
  T? result = null;
  foreach (var v in values)
    if (!result.HasValue || (v.CompareTo(result.Value) > 0))
      result = v;
  return result;
}
public static T max<T>(IEnumerable<T> values) 
  where T : class, IComparable<T>
{
  T result = null;
  foreach (var v in values)
   if (result==null || (v.CompareTo(result) > 0))
     result = v;
  return result;
}

only to find that

  • C# does not include generic constraints into method signatures, causing the code to not compile
  • Enumerable.Max() and Queryable.Max() are separate extension methods

In fact, Enumerable.Max() (which handles arrays, Lists, and everything IEnumerable) defines various overloads for base types, and one generic overload, whereas Queryable.Max() only defines the generic method. So the code above would have to be changed to base type-specific methods and one generic method.

Of course, the above code for IEnumerable would have no effect on how NHibernate handles the IQueryable.Max() method. This would have to be dealt with using NHibernate extensions.


Unknown Class “ProfileCommon”

January 19, 2016

I was asked to have a look at a Web Site project, and whether it could be converted into a Web Application project. So I created a web application project, copied and added all project files, and after a bit of editing (adding namespaces, etc.) I came across the error:

The type or namespace name ‘ProfileCommon’ could not be found (are you missing a using directive or an assembly reference?)

I soon found out that the class ProfileCommon is being automatically generated during the build process of a Web Site project, and in the meantime a VS BuildTask and a couple lines of code can be found on the internetz which replicate to original code generator.

To understand what it going on, I opened the original web site project, searched for a reference to ProfileCommon, and hit Go To Definition to retrieve the generated code.

As it turns out, the generator simply parses the configuration/system.web/profile/properties section of the web site’s web.config file.

Every item in the section contains a property definition in the form

<add name="[PropertyName]" defaultValue="[value]" 
    type="[.Net Datatype]" />

From this information, the generator creates a public class like this:

using System;
using System.Web;
using System.Web.Profile;
public class ProfileCommon : System.Web.Profile.ProfileBase {

Each declared property is generated as

  public virtual [Datatype] [PropertyName] {
    get {
      return (([Datatype)(this.GetPropertyValue("[PropertyName]")));
    }
    set {
      this.SetPropertyValue("[PropertyName]", value);
    }
  }

and finally

  public virtual ProfileCommon GetProfile(string username) {
    return ((ProfileCommon)(ProfileBase.Create(username)));
  }
}

My guess is that the ProfileBase.GetPropertyValue() method also evaluates the defaultValue clause somehow, but MSDN does not mention this question.


Compile-Safe Links in ASP.Net MVC

June 11, 2015

One of the drawbacks of the Html.ActionLink() method is that it accepts controller and action as strings. As a consequence, there no compile-time check is performed to ensure that the referenced action is actually implemented. This also applies to Html.BeginForm(), Url.Action() and similar methods.

Recently I found an alternative extension method on SO which accepts the controller class and a controller method as parameters, rather than their names as strings:

public static MvcHtmlString ActionLink<TController>(
    this HtmlHelper htmlHelper,
    Expression<Action<TController>> expression, 
    string linkText, 
    object routeValues, 
    object htmlAttributes) where TController : Controller
{
    var urlHelper = 
        new UrlHelper(htmlHelper.ViewContext.RequestContext, 
            htmlHelper.RouteCollection);

    var attributes = AnonymousObjectToKeyValue(htmlAttributes);

    var link = new TagBuilder("a");

    string actionName = ExpressionHelper.GetExpressionText(expression);
    string controllerName = 
        typeof(TController).Name.Replace("Controller", "");

    link.MergeAttribute("href", 
        urlHelper.Action(actionName, controllerName, routeValues));
    link.MergeAttributes(attributes, true);
    link.SetInnerText(linkText);

    return new MvcHtmlString(link.ToString());
}

private static Dictionary<string, object> AnonymousObjectToKeyValue(
    object anonymousObject)
{
    var dictionary = new Dictionary<string, object>();
    if (anonymousObject == null) return dictionary;
    foreach (PropertyDescriptor propertyDescriptor in 
        TypeDescriptor.GetProperties(anonymousObject))
    {
        dictionary.Add(propertyDescriptor.Name, 
            propertyDescriptor.GetValue(anonymousObject));
    }
    return dictionary;
}

Personally, I prefer the single-action controller approach, as sketched here, or here, for example, where each action is implemented by a separate controller class. The meaning of the ActionLink() parameters changes such that the controllerName parameter is really a sub-namespace, and actionName is the controller class name.

Then we can simply extract the last part of the namespace and the class name of the controller class name, and pass everything else to MVC’s Html.ActionLink():

public static MvcHtmlString ActionLink<C>(this HtmlHelper htmlHelper, 
    string linkText, 
    object routeValues = null, 
    object htmlAttributes = null) where C: BaseController
{
    if (string.IsNullOrEmpty(linkText))
        return new MvcHtmlString("");

    var rgs = typeof(C).FullName.Split('.');
    return htmlHelper.ActionLink(linkText, 
        rgs[rgs.Length - 1].Replace("Controller", ""), 
        rgs[rgs.Length - 2], 
        routeValues, htmlAttributes);
}

 


Avoiding repetitive code using Actions and Funcs

March 12, 2015

Some coding guidelines require that functions follow a pre-defined scheme, such as

  • log function begin
  • execute function
  • log function end
  • exception handler
  • log exception

to name just the simplest structure.

Since repetitive code is prone to errors, it’s a good idea to encapsulate these steps in a method which in turn calls the “execute function” step.

An example from an MVC project:

public ActionResult Method() {
  logger.Info("begin method");

  try {
    var user = CurrentUser();
    var data = DoSomething(user);
    logger.Info("end method");
    return View("View", data);
  }
  catch (Exception ex) {
    logger.Error("method exception", ex);
    throw;
  }
}

We can now extract most of the code into a common method

protected ActionResult Execute(string actionName, 
  Func<User, ActionResult> action) {
  logger.Info("begin " + actionName);

  try {
    var user = CurrentUser();
    var result = action(user);
    logger.Info("end " + actionName);
    return result;
  }
  catch (Exception ex) {
    logger.Error(actionName, ex);
    throw;
  }
}

so that the original method can be reduced to

public ActionResult Method() {
  return Execute("method", (user) =>
  {
    var data = DoSomething(user);
    return View("View", data);
  });
}