Realestate analysis with MapBox and Turf

Couple months ago I was thinking about buying a flat in Prague. That is pretty bold decision and requires some analysis before. Before looking further into the market I was looking for a web that would give me the average prices per meter in different city regions. I didn’t find anything and since I have always liked building stuff around maps I have created a simple web page. Everything is in JavaScript besides the backend part for gathering the data, which is F#.

Read More

Options and Stocks charts in F# and JavaScript

I have recently been playing with options, their pricing and pay-off charts generation. I have created a small library called Pricer which can do several thing like option price calculation, generation of data for payoff charts or volatility estimations for stocks. In order to demonstrate what the library can do, I have created a small web application Payoffcharts.com. The web site contains few different visualizations, payoff charts of arbitrary strategy based on options being one of them. The whole project is available on GitHub.

Read More

ElasticSearch TopHits with FluentNest

When providing statistics or charts to the users, one usually groups the values into aggregated collections. Let’s say sum per month, or average per category. In certain cases you might want to provide few details about the most significant elements in each bucket. For instance what were the biggest trades contributing to the total for given month. This can be easily done with ElasticSearch, but the queries get quite complicated. This post shows an easy way using FluentNest library.

Read More

Options pricing in F#

I have recently written a small option pricing library in F# called Pricer (what an original name…). It contains a set of methods for derivatives pricing, generating payoff charts, estimating volatility. Payoff charts show you the profit of an option as a function of the share price change. One can create a payoff charts of any derivative. Multiple options traded in the same time form strategies which can be used to obtain different type of protection or leverage against the movements of the underlying stock - but that is not the subject of this post.

Read More

JSON serialization with F#

This post is about F# types serialization using Json.NET library. Luckily enough Json.NET handles most of the F# types correctly but there are still few issues. Typically you can have problems with the F# specific types. Luckily enough there are quite a lot of examples on the internet of convertors for each of these types, that one can just plug into Json.NET. Here is a short list with an example of convertor that one can find on the web.

Read More

Paket referencing GitHub files

Paket is a dependency manager for .NET projects, besides referencing NuGet packages, one can reference as well any file from GitHub. Any file from GitHub just by adding single line to paket.dependencies files. Such files would however be added as links to the solution and if they should be picked up by IIS or other web server they need to be copied to it’s location in the solution. Here is how to make sure the files will be copied to their location.

Read More

F# folder rendered multiple times

I was recently testing the ASP.MVC 5 F# template, which works fine but almost always breaks down as soon as you add new item to the solution. F# has hard time with folders. The problem has been described here . And as soon as you add new file to the F# project you will run to the same problem and the next time, you won’t be able to load the project file with:

Read More

C# Fluent Interface for ElasticSearch

NEST already provides a Fluent like interface for querying ElasticSearch, but to my taste this query language stays too close to ElasticSearch JSON query format. The result is reduced readability of NEST queries and too much technical noise. I have come up with set of extensions methods which just wrap NEST and improve the readability and add a bit of expressiveness (by my biased judgement of course).

Read More

Financial Derivatives

When I started to work for my current company, I have struggled to get one simple overview of this universe. Instead I had to consult many sources, lot of them were going deep into details of one particular financial product. An I was not and still I am not interested in details of investment banking but just needed simple overview.

Over the time I took notes, and kept them for myself, getting information mostly from wikipedia or investopedia. I have also wrote some F# code to generate payoff charts for arbitrary options, which are used bellow:

Read More

Neural Networks F#, XOR classifier and TSP Hopfield solver

It seems that recently thanks to the buzz around Deep Learning, Neural Networks are getting back the attention that they once had. This post contains just a very short introduction to Neural Networks, just enough to understand the F# code which follows. The aim for me was to learn F# and see how easy it can be to write NN code in F#. Two usage of Neural Networks are presented here: XOR classifier (Hello World of NN) and Traveling Salesman Problem solved using Hopfield Neural Network. Both working examples are at GitHub

Read More

FSharp.Data and Unit Testing

I have recently run into two separate issues while testing some F# data providers based code. Both issues were linked to the FSharp Data assembly. I am using ReSharper’s NUnit runner and sometimes NCrunch. One of the problem was linked to the availability of FSharp.Data.DesignTime.dll on the compile time and other to NUnit not correctly handling Portable Library Class projects. I have tested and had the same issues with FSharp.Data 2.1.0 and 2.2.0.

NCrunch issues

For the first issue (missing FSharp.Data.DesignTime.dll) the fix depends on whether you are using Paket or Nuget to reference the package FSharp.Data package.

NCrunch won’t compile your solution when FSharp.Data is referenced. This component internally references FSharp.Data.DesignTime which has to be available for the compilation - and NCrunch does not have those libs available, because the DLL is not referenced the standard way, but must be provided by Visual Studio.

The current solution is to reference FSharp.Data.DesignTime manually. If you are using Nuget, than the dll can be found in the packages folder as shown bellow:

reference_fsharpdata

If you are using Paket then you can locate in the project file, the following reference to FSsharp.Data.

<Choose>
  <When Condition="$(TargetFrameworkIdentifier) == '.NETFramework' And ($(TargetFrameworkVersion) == 'v4.5' Or $(TargetFrameworkVersion) == 'v4.5.1' Or $(TargetFrameworkVersion) == 'v4.5.2' Or $(TargetFrameworkVersion) == 'v4.5.3' Or $(TargetFrameworkVersion) == 'v4.6')">
    <ItemGroup>
      <Reference Include="FSharp.Data">
        <HintPath>..\packages\FSharp.Data\lib\net40\FSharp.Data.dll</HintPath>
        <Private>True</Private>
        <Paket>True</Paket>
      </Reference>
    </ItemGroup>
  </When>
</Choose>

And you can place a similar reference for the DesigneTime dll just bellow:

<Choose>
    <When Condition="$(TargetFrameworkIdentifier) == '.NETFramework' And ($(TargetFrameworkVersion) == 'v4.5' Or $(TargetFrameworkVersion) == 'v4.5.1' Or $(TargetFrameworkVersion) == 'v4.5.2' Or $(TargetFrameworkVersion) == 'v4.5.3' Or $(TargetFrameworkVersion) == 'v4.6')">
      <ItemGroup>
        <Reference Include="FSharp.Data.DesignTime">
          <HintPath>..\packages\FSharp.Data\lib\net40\FSharp.Data.DesignTime.dll</HintPath>
          <Private>True</Private>
          <Paket>True</Paket>
        </Reference>
      </ItemGroup>
    </When>
  </Choose>

Notice that you can also remove the Paket tag. Removing this tag will assure that Paket will ignore this reference and won’t remove it while running paket install.

NUnit runner issues

NUnit did load the test correctly, but the test failed with SystemMissingMethodException

Luckily the solution can be found on StackOverflow and was caused by the tested library being Portable Class Library instead of standard F# library.

Read More

Events visualization with EvenDrops and KnockoutJS

I have recently needed to visualize a set of events which occurred within a certain interval. Each event would have couple parameters and there would be multiple event lines. Let’s say that you want to visualize the occurrences of car sales in couple countries. For each sale you would also want to visualize the price and the mark of the sold car. Before writing everything from scratch, I have found EventDrops project which responded to the majority of my requirements. It had just one flaw and that is that there is no way to chart another characteristics for each event.

Read More

Unit testing Knockout applications

In ideal case any View Model in Knockout based application should be completely unit-testable. The View Model of course interacts with other code but in majority of cases this would be either some UI code or the server side code, probably over REST API. The UI interaction should be minimal. If possible, the binding capabilities of Knockout should be leveraged. The REST API is not available while unit testing and thus has to be mocked or hidden by an abstraction layer.

Read More

Detecting extreme values in SQL

In a set of data points, outliers are such values that theoretically should not appear in the dataset. Typically these can be measurement errors or values caused by human mistakes. In some cases outliers are not caused by errors. These values affect the way that the data is treated and any statistics or report based on data containing outliers are erroneous.

Read More

Introduction to GPU programing through AMP C++

Since couple years CUDA (*2007) and OpenCL (*2008) have established themselves as standard frameworks for parallel programming on the GPU. In 2011 new framework came to the landscape of GPGPU programming, which does not compete on the same level, but rather represents new abstraction between the GPU and the developer.

Read More

Screen scraping in C# using WebClient

This post is intended to give you some useful tips to perform screen scraping in C#. In the ideal every solid web site, application or service should propose a decent API to provide the data to other applications. If the application holds resources of it’s users, than it should propose OAuth protected API and thus allow the users to use their data through another application. But since were are not there here are some tips screen scrapping tasks: authentication, state-full web applications, browser headers and others.

Read More

Single page app with RavenDB, KnockoutJS and Bootstrap

While learning a new technology or framework, I always like to build small but well covering Proof Of Concept application. It is even better if one can combine several new technologies into such a project. This is description of one such project. Single page web app using RavenDB, WebAPI, KnockoutJS, Bootstrap, D3JS. Available on GitHub

Read More

Playing with Google Drive SDK and JavaScript

I am just starting to use the Google Drive SDK for one of my personal projects. The front-end of the application is written entirely in JavaScript. I am in the process of integrating Google Drive and it took me some time to get throw the API reference and get it to work, here are some useful code snippets.

Read More

Introduction to Fakes and migration from Moles

Fakes is a new test isolation framework from Microsoft. It is inspired by and resembles to Moles a framework which I have described in one of my previous blog posts. In this post I will briefly describe Fakes and than show the steps which have to be taken when migrating from Moles. You will find that the migration itself is not that complicated. Besides some changes in the structure of the project only few changes are needed in the code.

Read More

Reflection & Order of discovered properties

In .NET Reflection provides several methods to obtain information about any type from the type system. One of these methods is GetProperties method which retrieves a list of all the properties of a given type. This method returns an array of PropertyInfo objects, but the order of these properties is not guarantied to be the same.

PropertyInfo[] propListInfo = type.GetProperties();

In most cases you don’t care, but the order of the properties does not have to be the same if you run this method several times. This is well described in the documentation of this method. Microsoft also states, that your code should not be depending on the order of the properties obtained.

Read More

Programming languages for the age of Cloud

This posts talks about the aspects which are influencing computer languages these days. We are in the age when the sequential execution is over. Even your laptop has a processor with several cores. The cloud provides us with tons of machines which we can use to run our code on. We are in the age of distribution, parallelization, asynchronous programming and concurrency. As developers we have to deal with the challenges which arise from this new environment. Computer language scientists have worked on the subject since the seventies. Nowadays concepts which have been studied for long time, influence the mainstream languages. This post describes how.

Read More

Common.Logging and compatibility with other libraries

It has been the second time since I have run into the issue of configuring correctly Common.Logging on my project. So what is the problem? Let’s start with the basics:

Common.Logging should be a generic interface for logging which can be used by other frameworks and libraries to perform logging. The final user (you or me) uses several frameworks in his final application and if all of these frameworks will use different logging framework it will turn into configuration nightmare.So our favorite frameworks such as Spring.NET, Quartz.NET are using Common.Logging. This interface in turn uses a concrete logging framework to perform the logging (the act of writing the log lines to somewhere).

Typical scenario can be for instance Common.Logging and Log4Net combination. In our application configuration file (web.config or app.config) we have to configure Common.Logging to use the Log4Net and than we can continue with the Log4Net configuration specifying what should be logged.

<common>
<logging>
  <factoryAdapter type="Common.Logging.Log4Net.Log4NetLoggerFactoryAdapter, Common.Logging.Log4Net">
 <arg key="configType" value="INLINE" />
  </factoryAdapter>
</logging>
</common>

<log4net>
<appender name="ConsoleAppender" type="log4net.Appender.ConsoleAppender">
  <layout type="log4net.Layout.PatternLayout">
 <conversionPattern value="%date %-5level %logger - %message%newline"/>
  </layout>
</appender>
</log4net>

My general problem is that Common.Loggin.Log4Net facade is looking for a concrete version of the Log4Net library. Concretely the version: ‘log4net (= 1.2.10)’. That is not a problem if you are not using some other framework which depends on higher version of Log4Net. In my case the le_log4net library (the logentries library) is using log4net 2.0. So if you are using NuGet, you might obtain the following exception while adding the references:

image

The similar thing might happen if you just decide to use the latest Log4Net by default. Then you might get an exception when initializing Spring.NET context or starting the Quartz.NET scheduler:

Could not load file or assembly ‘log4net, Version=1.2.0.30714, Culture=neutral, PublicKeyToken=b32731d11ce58905’ or one of its dependencies. The located assembly’s manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

Solution 1: Ignore NuGet, define Runtime Binding

One way to get around this is to define runtime assembly binding. But this solution forces you to add the reference to log4net manually. NuGet controls the version and wont let you at references on the fly the way that you would. So to get over add the latest Common.logging.Log4net facade and Log4Net version 2 (which you need for some reason). Than you have to define the assembly  binding in the configuration file.

<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
  <dependentAssembly>
 <assemblyIdentity name="Common.Logging" publicKeyToken="af08829b84f0328e"/>
 <bindingRedirect oldVersion="1.2.0.0" newVersion="2.0.0.0"/>
  </dependentAssembly>
</assemblyBinding>
</runtime>

Solution 2: Just use the older version of Log4Net (1.2.10)

If you do not have libraries that are dependent on Log4Net version 2.0.0 than just remember to always use log4net 1.2.10. This is the version which Common.Logging.Log4Net is looking for. Or just let NuGet manage it for you. You can add Common.Logging.Log4Net via NuGet and it will automatically load the correct version of Log4Net.

Solution 3: Try other logging library for instance NLog

This actually is not a real solution. I have experienced similar issues while using NLog, concretely try to use the latest NLog library with the Common.Logging.Nlog façade and you will obtain something similar to:

{“Could not load file or assembly ‘NLog, Version=1.0.0.505, Culture=neutral, PublicKeyToken=5120e14c03d0593c’ or one of its dependencies. The located assembly’s manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)”:”NLog, Version=1.0.0.505, Culture=neutral, PublicKeyToken=5120e14c03d0593c”}

The solution here is similar, you will have to define Runtime Binding:

<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
  <dependentAssembly>
 <assemblyIdentity name="NLog" publicKeyToken="5120e14c03d0593c" culture="neutral" />
 <bindingRedirect oldVersion="0.0.0.0-2.0.0.0" newVersion="2.0.0.0" />
  </dependentAssembly>
</assemblyBinding>
</runtime>

What was interesting here, is that NuGet actually took care of this for me. I have just added the Common.Logging.NLog façade and I guess NuGet spotted that I have already NLog 2 and that this Runtime Binding is necessary. If you look at the documentation of bindingRedirect you will see, that we have the right to specify the range of versions in the oldVersion attribute. Here all the version will be bound to the 2.0.0.0 version.

Summary

Anyway NLog and Log4Net are both cool logging frameworks, just use the one you prefer. As I have showed  above it is possible to use them together with Common.Logging it just takes a few more lines to configure it correctly.

Read More

KnockoutJS and Google Maps binding

This post describes the integration between Google Maps and KnockoutJS. Concretely you can learn how to make the maps marker part of the View and automatically change it’s position any time when the ViewModel behind changes. The ViewModel obviously has to contain the latitude and longitude positions of the point that you wish to visualize on the map.

Read More

Collective Intelligence: Ants colony solving TSP

This article is a description of TSP solver written in Python, using ants colony inspired algorithm. The algorithm is inspired by the behavior of ants and the way that they use pheromones to communicate. It belongs to larger group of algorithms inspired by Collective Intelligence.

Read More

Graph theory in Latex

For one of my previous posts, I needed some images of graphs. Initially I have taught, that I will just draw them in Inkscape or some other tool, but after a while I have decided to do something more clever – which might maybe serve me in the future – draw the graphs in Latex.

Read More

Invalid SystemRoot variable

This is a story of what happens when you set SystemRoot to incorrect recursive value. You can almost throw the PC out, because everyone depends on this variable. Until you realize that the variables are also stored in registry.

Last week I had experienced this quite “funny” situation on WinXP machine. When I have run some post-build scripts using Jenkings & Maven I have obtained something like:

Read More

Applications of Graph Theory

Graph Theory is just a beautiful part of mathematics. Not only Computer Science is heavily based on Graph Theory. There are a lot of applications of Graph Theory in Operational Research, Combinatorial Optimization, Bioinformatics. For my personal clasification I have separated the tasks, which you can solve using Graph Theory into two groups:

Read More

JavaScript asynchronously uploading files

At the beginning I have thought, that it has to be easy; just make a POST to the server using jQuery and the only question is how to get the data. Well I have found out that it is not that easy and googling around I have found there are quite a lot of pre-build components and plugins, which makes it quite difficult to decide for one of those.

Why it is not possible to use simple JavaScript POST?

Because of the security restrictions. The browser is not allowed to post the file content asynchronously. This is however about to change thanks to HTML 5.

Read More

Choosing technologies for .NET project

Our latest research and development project was an online banking application. While choosing the building pieces of this application, we tried to pick the State-Of-Art frameworks and technologies. This is not an easy task, while there are always several alternatives for each component. I have decided to created this post which sums up the technologies available for different parts of application and I will try regularly update it, to keep up with the changes.

Read More

Art of teaching computer science

I was studying for my final masters exams at CTU Prague, that means revising more or less all my previous studies. Thanks to some job experiences, that I had before, I am more or less able to see if what I have learned will be useful for me in the future and also what was missing during my studies.

Read More

Bind-able layer for Bing Maps

I needed to show a collection of collections of objects - in this case bike routes. Actually each route was composed of collection of routes (which could be interconnected at some places). This cannot be achieved only by MapItemsControl - which can render only one dimensional collection.

Read More

Catalina - Life cycle exception

This post is about maven downloading icu4j.jar as transitive reference to Jaxen and Tomcat Catalina not correctly starting after the reference has been added.

Read More

DotNetOpenAuth and Url rewriting (in Azure)

I have created a simple OAuth provider for my web application using DotNetOpenAuth. I have described how this has been done in my last post. Locally everything worked just fine. However when I published the solution to Azure then I obtained errors while processing authorized request.

Read More

Using DotNetOpenAuth to create OAuth Provider

DotNetOpenAuth is an open source library created and managed by Andrew Arnott which gives you the possibility to use OAuth protocol, OpenID and ICard. It si powerful - and comes with a nice Samples package. Recently I needed to implement OAuth provider, in other words I wanted to allow third party application to obtain data from my application after the user authorizes them to do so. Source code is available on GitHub

Read More

Provide JSONP with your WCF services (using .NET 3.5)

This post contains a small correction to MS provided example for returning JSONP with over WCF. The example is available here and works in most cases correctly, except the case when you are returning a raw JSON, that is you are not returning object which is serialized in to JSON, but rather returning a Stream which represents this JSON.

Such case might occur if you are building the JSON dynamically and not just serializing objects. The exception which you might obtain will be something along these lines:

Read More

Consuming WCF Services with Java Client

Here is the state of my latest project: I have a Silverlight application which talks to traditional WCF services in backend. The services have so far been configured automatically - so let’s say Visual Studio took care of the web.config. Newest requirement to my application was to allow Java clients consume these services.

Read More

ASP.NET Forms Authentication and Java client

This post describes my later situation. I have a Silverlight application which talks to traditional WCF services in backend. The services have so far been configured automatically - so let’s say Visual Studio took care of the web.config. Newest requirement to my application was to allow Java clients consume these services.

Read More

Silverlight Event Calendar

For one of my latest project I needed a quite simple Event Calendar component for Silverlight. I did not want to use any third party libraries and I wanted this component to stay simple. The code is available on GitHub

Read More

Silverlight DataForm ItemsSource is null when DataForm is not visible

This is just a quick post about Silverlight’s DataForm. When DataForm is not visible, then its ItemsSource collection is null. This can make sense, but it may also cause problems in certain cases when one would expect the collection to be filled.

Imagine the following: The DataForm is in Expander. On a click of a button you want to expand the Expander and add new item to the DataForm:

Read More

Bing Maps - Bindable TilesLayer

Specifying secondary tiles source allows you to cover parts of the map with you another layer. This layer might contain some additional information, such as new routes or points or any other geolocated information. In Silverlight you can use secondary tiles source by using the TilesLayer and LocationRectTileSource elements.

Read More

TFS and crashing QTagent32

I made a bug in my code which resulted in never-ending recursion which was produced only while UnitTesting. This might result into QTAgent32 crash locally. However if you have previously committed such code to TFS you might have to restart VSPerfMon process on TFS server.

Read More

Spring MVC tutorial

This semester I had to do a bigger project using Spring Framework & Hibernate. On a side of this project I had a smaller one which I have created just to always try out certain parts which later I applied to the bigger one. Here I just present the simple project which should be easy to understand, but it contains all the important parts/frameworks/design patterns of enterprise applications: MVC, ORM, Validations, Security, Internalization, Views/ Tiles, Unit Tests.

I do not have much experience working with J2EE, or writing enterprise Java applications, so this will maybe help someone who is on the same Java level as I am.

Read More

ReadOnly instance of NumberFormatInfo

When treating cultures, symbols and unites NumberFormatInfo.InvariantInfo is an easy way to get the Invariant culture. This instance however is read-only and can’t be modified. If you attempt to modify it you will obtain System.InvalidOperationException: Instance is read-only exception. This took me by surprise and caused my unit tests to fail and so I dug a bit deeper into the issue to understand how the read-only instance is created.

Read More

CORBA - Java Naming Service, manipulating contexts

This is a short post describing how to implement simple tool to manage CORBA Naming Service.

Corba

The aim of this paragraph is not to describe CORBA, but just to explain little bit how it works, so that later I can explain what is and how works Naming Service - which is part of the CORBA architecture.

So CORBA stands for Common Object Request Broker Architecture. It is a standard which enables programmers to write distributed applications, we can simplify it a lot and say it enables us to write apps working over the network.

Corba is just implementation of standard Server - Client architecture. In Corba’s world we create objects on the Server side - and these objects can be used by the clients, which are running on another computers on the network.

The client has to know the structure of the object (the methods and the attributes). The objects which are created on server side and used by the clients are before described by the IDL - interface definition language. The very big advantage of Corba is that if we describe the object by IDL we can later generate Java code as well as C++ code.

Read More

RSA implementation using GMP library

Right now I am studying in Paris in one of the engineering schools here and one of my last assignments of cryptography was to implement the RSA in C. To achieve this and to allow manipulation of big integers I have used GMP library which is an open source library for arithmetic.

Read More

J2EE - NetBeans, JSF, Persistence API

This is a very old article from the times, when I was very new to Java world. Nowadays [2013] I would probably not recommend looking into JSF and J2EE, but it can serve well someone who decided for these technologies and wants to get started. This article is divided into 3 parts:

Read More

Shrew VPN Client Windows 7 - 64 bit

Like many other people on the internet I had to use Cisco VPN Client to connect to our network. First Cisco didn’t support 64 bit, now they do support, but I am unable to install the 64bit client. I get this error during the installation:

Read More

Building Midnight-Commander

I was trying to compile Midnight-Commander from source and get it to run. I downloaded the source files using Git and than went to the project folder and run the ./configure script. Because it was on my newly installed OpenSuse machine I run into following errors and solved them by installing required components:

Read More

Sharepoint, workflows, infopath and headaches

My first partial job was in a company that was heavily rallying on SharePoint. My task was to write custom workflows using the Workflow Foundation and ASP.NET. I was a very bad programmer than and there was absolutely no development culture in the place that I worked. I have started to blog at the same time about my daily fights with SharePoint. I have merged the old posts into single one and I leave them up here for two reasons. If any pour programmer works with these technologies, he might find a fix here and less suffering. And I also like to look back at what I did as beginner and how I have evolved.

Read More