Probability Distributions

Yes I know…what a thrilling blog post title! But you know, sometimes I get the feeling we get all caught up in overly complicated ways to solve problems, its neural nets that, boosting this, recommender that, yada yada, yada….like whatever!

We forget that in data analysis its so important it to have an intuitive grasp on the basics. And with that in mind I just had to post this amazing image from Sean Owens great blog post on understanding the relationship between different probability distributions. I started searching for something like this because I got into this long and thrilling conversation the other day on different distributions and everyone seemed a little too vague about it all. I tell you, I felt uneasy!

So check out the post and ensure you have a zen understanding of how they they all fit together!




One of things about MusicXML is that it does not explicitly encode time information, but just sticks everything in a list. You can’t extract any particular note or rest from the data in order to find its exact position.

To solve this I have created a node module that takes a MusicXML file and converts it into MusicJSON! This provides all the time information you would expect such as tempo, the absolute location, the location in a measure of a note or rest, the location of the beginning of measures, and an ISO8601 compliant timestamp location in milliseconds that is calculated using location and tempo information.

Having this kind of information encoded in JSON makes data visualisation far simpler and also makes the data set well suited to a range of machine learning and machine intelligence applications.

I have included a pianoroll visualization express app in this node module (which generates the visualization below), and is the test I have found most handy. You can check it all out on my github at



Angular research template

After using all kinds of editors over the years, from Emacs to Word to LaTex, I have to say there has never been a better time to be doing research. Tools like Jupyter Notebook and the MEAN stack mean that you can easily have access to an industrial weight data store right along side your academic writing. There is nothing cooler than having access to variables in academic writing, and being able to put together interactive visualisations so people can immerse themselves in your data when they are reading your stuff (well, ok, there is probably some cooler stuff than this…but even so).

But while this brave new world is cool, sometimes you do need to able to print stuff out. If you work across different disciplines especially, its not enough to have all your stuff living on the web. You need to be able to create high quality, camera ready papers suitable for hard copy printing.

To solve the issue, I created a cool angular research template. Styled for even the most discerning tweed wearing academic type, its a way to work in the web environment with a view to printing.

Screen Shot 2016-08-08 at 10.57.00 AM

The features?

  • A simple CSS set up to manage A4 pages and margins, fonts, and some Twitter Bootstrap styling, super handy for basic layout stuff.
  • Angular driven website under the hood, so all the advantages that come with that. You can beautifully organise your data in factories and all that other Angular garb. Directives, along with the CSS, give you that nice LaTex style control that we all love and crave.
  • I have included some other cool javascript libraries I always use like D3 for data visualisation and MathJax for beautiful math typesetting

You can check it out on my Github (which is another neat thing of this set up – simple version control!).


Brad Osgood’s Fourier Lectures

The theories of Fourier are just such an astounding piece of thinking. How did he come up with this stuff? How did Euler pave the way for him? Fourier makes us realise that  mathematics , though bounded by scientific rigour, is a profoundly creative endeavour, driven above all by intuition and outlandish ideas.

I have been thinking about this a lot lately, while rewatching  Brad Osgood’s amazing lectures on the Fourier Transform.  Not only are Osgood’s lectures just so cool, but his notes on the lecture series are great too. Math texts are almost always badly written so these were an amazing find.

Now, in these inter-disciplinary times how do we find the time for Fourier? To get into them make sure your Calculus is under control, and maybe some Linear Algebra too.

And just in case you are all like, why on earth would you bother with Fourier?

Because it is worth it. Fourier changes everything.