Visualizing ECDC’s COVID-19 data

The ECDC (European Centre for Disease Prevention and Control) is providing world-wide data on Coronavirus infections and deaths.

The data format changed a couple of times, beginning with date-stamped Excel files (.xlsx), then .csv files.  then the column names  and as of today, no more .xlsx files are provided.

(I initially tried to analyze the .xlsx files with SheetJS, but always got the error message

Cannot find file [Content_Types].xml in zip

which I could not solve, therefore switching from .xlsx to .csv).

As file hosting moved to opendata.ecdc.europa.eu, column names changed (at least in the .csv), and the date has been removed from the filename.

So I started out with a little pen on CodePen.io (data as of March 25), consisting of loading data from .csv, selecting countries of interest, and the kind of data to be displayed in a Chart.js chart:

  • Number of cases, or number of deaths
  • Count per day (as directly from the data files); total sum per day; and change in cases per day, optionally averaged over a number of days

After experimenting with the pen, I exported it and created the full HTML page containing the original pen, and adjusting a couple of things.

Next, I created a Scheduled Task to fetch the latest ECDC data and store the file on the webserver. (Originally I had a problem with fetching the file directly in Javascript because the webserver did not provide a Access-Control-Allow-Origin header)

You can find the most current version of my Covid visualization on my website.

Solving SignalR $.connection.HubName is null or undefined

I added support for SignalR to one of the ASP.Net MVC applications I develop, mainly following this Microsoft tutorial.

I was surprised by the fact that you cannot install the SignalR package from the NuGet GUI (in VS 2013 at least), but need to switch to the Package Manager Console and type

install-package Microsoft.AspNet.SignalR

It worked nevertheless.

I also copied the JS initialization code from there

 $(function () { 
    // Declare a proxy to reference the hub. 
    var chat = $.connection.chatHub;
    chat.client.broadcastMessage = function (name, message) { ... };
}

(adjusted, of course, to the hubs defined in my project), only to find that the JavaScript console in Chrome Developer Tools displays the error message

Uncaught TypeError: Cannot read property 'client' of undefined
 at HTMLDocument. (38024:10393)
 at fire (jquery-1.12.4.js:3232)
 at Object.fireWith [as resolveWith] (jquery-1.12.4.js:3362)
 at Function.ready (jquery-1.12.4.js:3582)
 at HTMLDocument.completed (jquery-1.12.4.js:3617)

The error is raised by the line

chatHub.client.broadcastMessage = function (name, message) {

with chatHub being undefined.

I stepped through the JavaScript code and found that the (automagically generated) JavaScript file

< script src="@Url.Content("~/signalr/hubs")" >< /script>

defines the hubs correctly, but somehow the hub definitions are lost when the

$(function() {

is being executed.

It turned out to be a question of JavaScript scopes, and I moved the client callback definition out of the jQuery $(function(){}) directly into the

Debugging Ajax’ed JavaScript and jQuery val() calls

I develop a web application which displays data in a read-only form, and loads the edit form upon pressing a button

$(function () {
  $(".btnEdit").click(function () {
    $.ajax({
      url: '@Url.Action("Form", new { id = Model.ID })',
      type: 'post'
    });
  });
});

Now I needed to debug the JavaScript code loaded by the call to $.ajax(), but Chrome does not seem to display the loaded response in its tree of Sources.

An answer on SO provided me with the solution: Simply add the line

//# sourceURL=@Url.Action("Form", new { id = Model.ID })

inside a <script> block in the AJAX-loaded HTML. This will add the requested URL under the (no domain) node

chrome ajax js

Now that I have access to the source file, I needed to find all invocations of the jQuery val() function, since I was tracking down a wrong value in an <input>.

Again SO provided a solution, which I added to my code

(function($){
  var originalVal = $.fn.val;
  $.fn.val = function(){
    var selectorPath = $(this).parents().map(function () {return this.tagName;}).get().reverse().join("->");
    console.log("val( #" + $(this).attr("id") + " " + selectorPath + " , " + JSON.stringify(arguments) + ")");
    var result =originalVal.apply(this,arguments);
    return result;
  };
})(jQuery);

Now that the calls to val() were logged to the Console, it was easy to find where the wrong value was set.

TypeScript logging with log4javascript in an ASP.Net MVC application

I needed to add logging functionality to an existing TypeScript application hosted on ASP.Net MVC.

Since I usually use log4net in C#, log4javascript was an obvious choice, as it provides a similar API.

First, I needed to declare the parts of the log4javascript API I use in TypeScript:

declare module log4javascript {
    class Logger {
        info(msg: string);
        info(msg: string, value: any);
        addAppender(app: AjaxAppender);
    }
    function getLogger(name: string): Logger;
    function getNullLogger(): Logger;
    class AjaxAppender {
        constructor(url: string);
        setSessionId(id: string);
        setLayout(layout: JsonLayout);
        addHeader(key: string, value: string);
    }
    class JsonLayout {
        constructor(readable: boolean, combine: boolean);
    }
}

In the logging class, I added logger and AjaxAppender. Depending on the data to log, I also configure a JsonLayout.

logger: log4javascript.Logger;

A non-logging logger instance is created using getNullLogger():

this.logger = log4javascript.getNullLogger();

Plain-text AjaxAppender

Use these statements to create an Ajax-based logger in TypeScript:

this.logger = log4javascript.getLogger("MyLoggerName");
var app = new log4javascript.AjaxAppender("http://myhost/myapp/mylogger");
this.logger.addAppender(app);

On the MVC side, the controller receives the parameters: logger, timestamp, level, url, message, sessionid, and layout. (Note that these parameter names can be changed using the setKeys() method)

The value for sessionid is set using

ajaxAppender.setSessionId(sessionId);

To log a string message, simply write

this.logger.info("my info log message");

The C# controller method stub therefore looks something like this:

[ValidateInput(false)]
[HttpPost]
public ActionResult Log(long? timestamp, string level, string sessionid, string message)
{
    var ipaddr = Request.ServerVariables["HTTP_X_FORWARDED_FOR"];
    if (string.IsNullOrEmpty(ipaddr))
        ipaddr = Request.ServerVariables["REMOTE_ADDR"];
    if (string.IsNullOrEmpty(ipaddr))
        ipaddr = Request.UserHostAddress;

    if (level == "INFO")
        logger.Info(ipaddr + " " + sessionid + " " + message);
    else
        logger.Debug("[" + level + "] " + ipaddr + " " + sessionid + " " + message);
    return Json(null);
}

where logger is an instance of log4net.Ilog.

JSON-formatted AjaxAppender

To change the appender’s data format to JSON, we set its layout to JsonLayout:

this.logger = log4javascript.getLogger("MyJsonLogger");
var app = new log4javascript.AjaxAppender("http://myhost/myapp/myjsonlogger");
app.setLayout(new log4javascript.JsonLayout(false, false));
app.addHeader("Content-Type", "application/json");
this.logger.addAppender(app);

The C# controller method receives a JSON array of log entries

    [
        {
            "logger":"MyJsonLogger",
            "timestamp":1448621329758,
            "level":"INFO",
            "url":"http://localhost/myapp/...",
            "message":
            [    "message1",
                "message2", ...
            ]
        }
    ]

where message1, message2, etc are the parameters of the logger’s debug/info/warn etc. calls.

We create a C# class containing the required properties

public class Data
{
    public string logger;
    public long? timestamp;
    public string level;
    public string url;
    public string[] message;
}

and deserialize the HTTP request’s InputStream using JSON.Net:

[ValidateInput(false)]
[HttpPost]
public ActionResult Execute()
{
    string inputContent;
    using (var sr = new StreamReader(HttpContext.Request.InputStream))
    {
        inputContent = sr.ReadToEnd();
    }
    var data = JsonConvert.DeserializeObject<Data[]>(inputContent);

    if (data != null)
    {
        foreach (var d in data)
        {
            if (d.level == "INFO")
                logger.Info(d.message[0] + " " + d.message[1]);
            else
                logger.Debug("[" + d.level + "] " + d.message[0] + " " + d.message[1]);
        }
    }
    return Json(null);
}

Again, logger is an instance of log4net.Ilog.

Detecting Screen Orientation Change

Browsers provide different means to detect screen orientation:

Documentation in the Mozilla Developer Network (linked above) states the first to be deprecated but currently still in the WhatWG Living Standard, whereas its documentation on the latter differs from the W3C documentation.

According to documentation, detection of screen orientation change can be achieved by implementing handlers for the events

  • window.orientationchange
  • screen.orientation.change
  • window.matchMedia() listener
  • window.resize

but specific browsers may not support all of these events, with window.resize being the catch-all solution if everything else fails.

So based on SO answers and this blog and this blog I came up with a solution that currently seems to work, and a couple of findings:

  • window.orientation gives the angle on mobile browsers only – desktop browsers always contain 0 (zero).
  • Similarly, window.onorientationchange is only supported by mobile browsers.
  • screen.orientation (and its browser-spezific siblings mozOrientation and msOrientation) contains the angle in its angle property. IE11 does support support screen.orientation on Win7. Mobile Chrome (35) and the Android 4.4.2 Browser do not seem to support it either.
  • Of the browsers I tested, none seem to implement the event screen.orientation.onchange.
  • Orientation change can be detected using the window.matchMedia() listener on both mobile and desktop browsers which support mediaqueries and its orientation selector.
  • In desktop browsers, orientation can only be derived from $(window).width() and $(window).height(), or from the .matches property of a matchMedia listener.

Note that all this need not apply for older browsers, not even the values of window.orientation! (See SO, SO, SO, Giff’s note)

So here now is my JavaScript code for screen orientation change detection:

function doOnOrientationChange(src)
{
  if (window.console && console.log) 
    console.log("width " + $(window).width() + " height " + $(window).height());

  var orientation = { 
    angle: window.orientation,
    type: ("onorientationchange" in window) ? "mobile" : "desktop"  
  };

  if (window.screen) {
    var o = window.screen.orientation || window.screen.mozOrientation 
      || window.screen.msOrientation || orientation;
    orientation = { angle: o.angle, type: o.type };
  } else if ((window.orientation === 0) || window.orientation) {
    orientation = { angle: window.orientation, type: "" + window.orientation + " degrees" };
  }
 
  if (!("onorientationchange" in window)) {
    var w = $(window).width(), h =$(window).height();
    var a = (w > h) ? 90 : 0;
    orientation.angle = a;
    if (window.console && console.log) 
      console.log("angle := " + a + " " + orientation.angle);
  }
 
  var jsonOrientation = JSON.stringify(
    { angle: orientation.angle, type: orientation.type });

  switch(orientation.angle) 
  { 
    case -90:
    case 90:
      // we are in landscape mode
      $().toastmessage('showNoticeToast', src + ' landscape ' + " " + jsonOrientation);
      if (window.console && window.console.log) console.log(src + ' landscape ' + " " + jsonOrientation);
      $("#orientation").text(src + ' landscape ' + " " + jsonOrientation);
      break; 
    case 0:
    case 180:
      // we are in portrait mode
      $().toastmessage('showNoticeToast', src + ' portrait ' + " " + jsonOrientation);
      if (window.console && window.console.log) console.log(src + ' portrait ' + " " + jsonOrientation);
      $("#orientation").text(src + ' portrait ' + " " + jsonOrientation);
      break; 
    default:
      // we have no idea
      $().toastmessage('showNoticeToast', src + ' unknown ' + " " + jsonOrientation);
      if (window.console && window.console.log) console.log(src + ' unknown ' + " " + jsonOrientation);
      $("#orientation").text(src + ' unknown ' + " " + jsonOrientation);
      break; 
  }
}

$(function () {

  if ("onorientationchange" in window) 
    window.addEventListener('orientationchange', 
      function() { doOnOrientationChange("window.orientationchange"); });
  //window.addEventListener('resize', 
  //    function() { doOnOrientationChange("window.resize") });
  if (window.screen && window.screen.orientation && window.screen.orientation.addEventListener)
    window.screen.orientation.addEventListener('change', 
      function() { doOnOrientationChange("screen.orientation.change"); });

  if (window.matchMedia) {
    var mql = window.matchMedia("(orientation: portrait)");
    mql.addListener(function(m) {
      if (m.matches) {
        doOnOrientationChange("mql-portrait");
      } else {
        doOnOrientationChange("mql-landscape");
      }
    });
  }

  doOnOrientationChange("init");
});

(I put the window.resize handler into comments because it generates too may events on desktop browsers.)

In this sample code, detection change only causes output of angle and orientation type to

  • $().toastmessage() – a jQuery extension
  • console.log
  • $(“#orientation”).text() – a jQuery call

Of course, your handlers may perform some useful actions…

Invoking Custom JavaScript Code from Angular2

I am trying to migrate an ASP.Net MVC application to Angular2. One thing I am still missing is how to invoke custom/legacy JavaScript code, as Angular2 does not render <script> tags contained in templates.

Actually we do not need <script> tags, as custom JavaScript/TypeScript can be called from the @Component!

Custom code can be invoked from various places of your component’s TypeScript code:

Logging to the console shows that code is executed in the following order

  • constructor
  • ngAfterContentInit
  • ngAfterViewInit
  • asynchronous code invoked in constructor (e.g. by http.get())

(for more details, see Angular’s documentation of Lifecycle Hooks)

Since executed code resides in the @Component, we do not need to pass JavaScript objects to the template.

Whereas in ASP.Net MVC we have a strict distinction between Controller and View, and process and data flow are from the controller to the view, and the view cannot (usually does not) reference the controller, this distinction does not apply in Angular2, where we end up in the component after the template has been rendered.

Angular2 “Does not support more than 9 expressions”

If you need to pass more than just a couple of values from your Angular2 component to the template, you’ll sooner or later run into the error message

EXCEPTION: Does not support more than 9 expressions

I needed to do just that to provide TypeScript/JavaScript objects with a huge set of values. The solution to work around this restriction is to create an object in the component which stores all these values, and then access its JSONified value

export class MyComponent {
  o: any;

  constructor() {
    var o = { .... complex JavaScript object ... };
    this.o = JSON.stringify(o);
  }
}

from the template

< script >
  var o = {{ o }};
  ....
</script >

…except that <script> tags are not rendered in Angular2 templates.

We can still store the JSONified value in an HTML attribute. But to call custom or legacy JavaScript code, we can extend the @Component.

Retrieving Chrome tabs suspended by The Great Suspender

After a recent Windows crash, restarting Chrome would not open all the windows and tabs I had collected, despite its nice “Restore previous session” question.

None of the tips I found on the net helped recovering those tabs:

Anyway, I found that the tabs suspended by The Great Suspender were most likely not contained in the Tabs file, so where is that information stored?

By looking at the source it turns out that TGS stores its data in an IndexedDB.

To access this database, you need to open TGS’s extension URL chrome-extension://klbibkeccnjlkjkiokjodocebajanakg/suspended.html. Using F12 (Developer Tools) and selecting Resources, you find “tgs” under IndexedDB, containing the tables gsCurrentSessions, gsSuspendedTabInfo, gsPreviews, and gsSavedSessions.

Unfortunately, currently there is no way to export this data directly from the DevTools, so we need a little bit of scripting magic.

After copying and pasting from MDN’s IndexedDB API documentation and sample code and some trial and error, I finally completed a small Javascript script which, when entered in the Developer Tools’ Console, results in a JSON-formatted list of all suspended tabs:

var db; var result=""; var record=0;
var dbor = window.indexedDB.open("tgs");
dbor.onsuccess = function(event) { 
  db = dbor.result;
  db.transaction(["gsSuspendedTabInfo"], "readonly")
    .objectStore("gsSuspendedTabInfo")
    .openCursor()
    .onsuccess= function(event) {
      var c=event.target.result;
      if (c){
        result+=JSON.stringify(c.value)+"\r\n";
        console.log(record++);
        c.continue();
      } else {
        console.log(result);
      }
    };
};

A single JSON record of a suspended tab looks like this:

{
  "date":"2015-12-14T18:40:55.563Z",
  "title":".validate() | jQuery Validation Plugin",
  "url":"http://jqueryvalidation.org/validate#toptions",
  "favicon":"chrome://favicon/http://jqueryvalidation.org/validate#toptions",
  "pinned":false,
  "index":3,
  "windowId":2898,
  "id":1281
}

Of course, you can process each tab’s record (c.value in the code above) any way you need.

Some Facebook Javascript Bookmarklets

Bookmarklets are pieces of JavaScript code that are stored as bookmarks in your browser, and execute locally (i.e. inside the downloaded page) as the bookmark is clicked.

Using the Facebook Graph API, you can take a look “behind the scenes” to retrieve the raw information of what is being displayed when you browse Facebook.

Requests under the URL http://graph.facebook.com/ return JSONified data about the requested object, which is identified by it’s ID.

Posts and Threads

Let’s look at threads (posts) and their IDs. There are a couple of ways the thread ID is stored in the thread URL, depending on where it is posted (page, group) and how you browse it (permalink or notification):

https://www.facebook.com/[name]/posts/[thread id]
https://www.facebook.com/groups/[group id]/permalink/[thread id]/
https://www.facebook.com/photo.php?fbid=[thread id]&set=[...]

Since the thread ID is numeric, a simple regex \d+ would be sufficient to retrieve it. However, group IDs may also be numeric, and names may contain digits.

After a bit of experimenting, the regex that I came up with to extract the thread ID from a Facebook URL, is

(/[\/=](\d+)[&\/]?/.exec(window.location.toString().replace(/\/groups\/\d+\//),""))[1]

Using this regex, we can now craft a JavaScript routine to open a new window containing the Graph API result:

window.open("http://graph.facebook.com/" +
    (/[\/=](\d+)[&\/]?/.exec(window.location.toString().replace(/\/groups\/\d+\//),""))[1] +
    "/comments", "_blank")

and create a bookmark for it. Since this WordPress installation does not allow to include bookmarklet links, you need to

  • Create a bookmark in your browser
  • Give it a name, such as “View Comments in FB Graph”
  • Set the URL or location to
javascript:window.open("http://graph.facebook.com/" + (/[\/=](\d+)[&\/]?/.exec(window.location.toString().replace(/\/groups\/\d+\//),""))[1] + "/comments", "_blank")
  • Click OK

But this does not give you the contents of the whole thread, just the comments.

To retrieve the whole post, we can use the Graph API Explorer.

The Graph API Explorer retrieves the details of a Facebook object, such as a post or thread, using the URL

https://developers.facebook.com/tools/explorer/?method=GET&path=[object id]

So, as we know how to extract the thread ID from a FB URL, let’s create a bookmarklet with the URL

javascript:window.open("https://developers.facebook.com/tools/explorer/?method=GET&path="+ (/[\/=](\d+)[&\/]?/.exec(window.location.toString().replace(/\/groups\/\d+\//,"")))[1], "_blank")

This opens the Graph Explorer with the desired ID. Click Submit to retrieve the data. Probably you need to click Get Access Token first.

Remove the Right Column

If you want to take screenshots of Facebook pages, you probably want to remove the right column before screenshotting, since it only expands the image, but does not include the content you want to save.

The top-most HTML container for right column content is called “rightCol” (yes, surprising).

To remove it from display, simply add this code to a bookmarklet:

javascript:var rc=document.getElementById("rightCol");rc.parentElement.removeChild(rc);

Clean up the Likes Page

To get a screenshot of “selected” Likes on a Like Page, there is a way to delete the Likes we don’t like (haha).

Simply scroll down until the list of likes is complete, then run this bookmarklet:

javascript:var li=document.getElementsByClassName("_5rz");for(var i=0;i<li.length;i++){var l=li[i];l.onclick=(function(el){return function() { el.parentElement.removeChild(el);return false;};})(l);}

Now clicking on a Like preview image will remove the entry from the list, allowing you to retain only the desired entries, ready to screenshot.

Using .resx Strings in Javascript

If you output a string in ASP.Net MVC using @ (Razor) or <%: %> (aspx), the string will automatically get HTML-encoded.

So if you reference a resource string (.resx) in Javascript,

<script type="text/javascript">
var x = "@resx.StringId";
</script>

AND the string contains characters the HTML-encoded (such as “&” => “&amp;”, umlauts, accented characters, etc.), you’ll end up with a lot of ampersands.

The way I solved this problem in a recent project was to define an HtmlHelper extension like this:

public static IHtmlString JsString<TModel>(this HtmlHelper<TModel> html, string s)
{
  return html.Raw(HttpUtility.JavaScriptStringEncode(s));
}

The string is first Javascript-encoded, and then written to the response stream unchanged using HtmlHelper.Raw().