Posts

An Erd web server: generation Entity Diagrams from a textual description with Haskell

Last week I started discussing how I am working on improving my approach to diagrams generation, so that it could become acceptable for a Software Engineer.

Currently I write a textual description of my entity-diagrams and I generate the corresponding images using Erd. Erd is a fantastic tool written in Haskell which parses its own DSL and invoke graphviz, producing images in several different formats. To use it you need to install Graphviz, the Haskell platform and cabal. This is not always straightforward, so I wanted to create a web interface for this program. This would offer a few benefits:

  1. I could then use Erd from wherever I want, without the need to install the toolchain
  2. I could later add Syntax highlighting and GitHub integration
  3. I would have an excuse good enough to play with Haskell before totally forgetting the few things I have learned

The basic idea is pretty simple: the web application will present an editor and when the user clicks on a button the code is sent to the server. The server process the code and generate an image which is sent back to the browser and displayed in the page. It should be enough for the MVP (Minimum Viable Product: now that the enterprise world has taken my soul I have to use these acronyms, right?).

I wrote this piece of software named Erd web-server and it is available on GitHub: https://github.com/ftomassetti/erd-web-server. In the rest of this post I describe its implementation.

Interfacing with Erd

Erd is a nice program written in Haskell. Now, ideally my project would just add Erd to the dependencies and everything should be easy and fine. Unfortunately it is not possible because Erd is a cabal application module (Cabal is the package manager for Haskell). It means that the moduled cannot be used as a dependency for other modules. So ideally I would like Erd to expose a library and, as a separate project, a console utility wrapping that library. I opened an issue in the Erd project and I hope to able to send a pull-request in the future. For now I just copied the code from Erd in my project. Basically I removed the Main function and I changed one single method (loadER) to accept a Text instance instead of a file handle. In this way I can receive the code in an http request and process it in memory, without the necessity of dumping it to a file.

Overview of the routes available

The Erd web server just present a web page where users can edit the description of the ER. Then the user can edit a button and that cause a post request to be sent at /generate with the code of the ER. If things go well an image is generated and it can be retrieved under /generated.

In addition to that we need to serve also assets (Javascript and CSS)

The viewIndex and generate functions

Serving the index is easy, we just read an HTML and we return it. In the future we could use some template system if we need to have some variable value.

The generate function contains the most interesting part of the code. It operates in the Snap monad from which we can access the request and prepare the response. We have to map IO operations using liftIO which basically translate IO X values in Snap X values.

As first thing we get the body of the request (getRequestBody) and we pass it to processErCode. processErCode could either succeed (if the ER code is correct) or fail:

  • if processErCode fails it return a Left String value, containing a description of the error. In this case we produce a response containing a JSON object with the field error, where we insert the error message
  •  if processErCode succeeds it return a Right ByteString value, containing the bytes of the actual image generated. Ideally we would like to return those bytes in the answer, for the MVP we just dump them to a file which we save locally and we return the name of the file. Note that we generate a random name for the file. This is not ideal and it require to clean up the directory of the generated files from time to time. It is also possible that someone overrides your file accidentally. In theory this solution is very, very poor. In practice it is super simple and it works well enough for my use case 🙂

Serving assets and generated diagrams

We just serve the contents of two directories:

  • assets here we store javascript and CSS files
  • generated this directory contains the diagrams generated at each POST request on /generate

Javascript code

Finally we need to some work on the client side, basically sending a post request to the server and process the result. It means either:

  • showing the error message in the status box
  • loading the image from the given URL

For implementing the call to the server I have used the promise.js library.

 

Screenshots

And now it is show time! From the following image you can notice two things:

  1. the application has a very simple interface: just the editor, the _Generate Diagram_ button, the generated image and a status label saying “Success!” in this case
  2. I am not the most fancy designer out there 🙂

 Screenshot from 2015-04-19 08:58:13

Final thoughts

There are many ways we could improve this application:

  • Update the image automatically when the user does not type for a few seconds
  • Report errors inline
  • Syntax highlighting and auto-completion for the editor

On the other hand I am really satisfied with the current result: with few lines of Haskell and Javascript we created an application that help me when working on those fancy diagrams, giving me more time to think about the concepts I want to represent and spending less time remembering the options to generate the diagram or installing tools.

Getting started with Frege: Hello World and basic setup using Maven

I spent a couple of hours playing with Frege (Haskell on the JVM) and not much documentation tutorials seems available. I am trying to help writing this simple Hello World tutorial.

The code is available on Github: https://github.com/ftomassetti/frege-tutorial/tree/01_HelloWorld

Update: Frege has some very useful documentation at http://www.frege-lang.org/doc/… where … represents the package, or module, name. For example, if one needs some reference for the frege.java.util.Regex package, one looks at http://www.frege-lang.org/doc/frege/java/util/Regex.html

Frege source code

The code is very simple for our little hello world example. In this tutorial we focus mainly on configuring our environment.

We declare the name of module to be HelloWorld. It will affect the name of the Java class produced.

The third line defines the type signature of the main function, while the fourth lines define main as a call to putStrLn using an IO Monad. In practice, you have to do the operations which affect the real world (like reading from a file or writing to the screen) inside a do statement. The reason is that the compiler treat them differently from pure functions, which can be optimized in several ways (lazyness, memoization, etc.) while “realworld operations” cannot.

Writing the POM (Maven configuration file)

First let’s take a look to the whole file:

The dependencies contain frege, no surprises here:

We then use two plugins, to compile Frege code and Java code:

Finally we save the classpath used by Maven in a file (classpath.conf) by using  the maven-dependency-plugin

The classpath.conf file will be useful for running the application using the run.sh script.

Running the application, the run.sh script

To run the application we need the frege jar and the classes generated from our frege source code.

Compile and running HelloWorld

After cloning the repository, you can simply run:

The result, should be something like:

[federico@normandie frege-tutorial]$ sh run.sh
Hello world. Frege is a lot of fun!
runtime 0.001 wallclock seconds.

 

Exploring frege: Haskell for the JVM

Recently I played a lot with Clojure and as part of my playing I built a civilizations simulator named civs. I really love building applications at the REPL and my Clojure code is much clearer and easier to read than the Java code I could have written for it

but…

when you need to refactor significantly your code it is going to be painful. Probably you can limit the issue with appropriate design choices and there is the possibility to use different forms of optional typing in Clojure, but still, you will miss a compiler helping you figure out what functions you need to revisit because of your small change in a data structure. Sure, I should add use different abstractions but…

So, I spent some time trying to figure out which functional static typed language was worthy to spend some time on, just to know better my options. I considered:

OCaml: it seems that the adoption is declining more and more according to different metrics and the current compilers do not support multithread, which seems a capital sin.

Scala: I have used it just to build an incremental parser for Java and I dislike it. It tries to be too many things at the same time and remember me C++, at the good old times when language designers taught that supporting more and more and more features in a language was a nice thing. That and the awful tool support made me decise that I do not want to touch it neither with a pole, unless forced too.

Haskell: what else is left? SML?

Now, as part of my job at TripAdvisor I am learning to be more and more practical and getting things done. I have already some investments done in the JVM environment. For example my world generator (lands) can be used through Jython on the JVM as well as the name generator (namegen) and a few other things that are written in Java or Clojure (no stuff left in JRuby, right?). So it would be great to have Haskell for the JVM…

…and then is when Frege comes to the rescue! It is just this, a port of Haskell to the JVM.

Basic toolchain

From the Github page you can download the jars you need. Visit this page: https://github.com/Frege/frege

You can play with the REPL and when you want to compile & run:

java -jar ~/tools/frege3.21.586-g026e8d7.jar main.fr
java -cp ~/tools/frege3.21.586-g026e8d7.jar:. Main

A small difference between Haskell and Frege

While in Haskell I would write:


data Name = String | Unnamed
deriving Show

 

In Frege I need to write:


data Name = String | Unnamed
derive Show Name -- no spaces at the beginning of the line. Yes, it matters

What derive means translated in Java? It means more or less I am too lazy to write the toString method for this class (Name) so please do the usual thing auto-magically and implement it for me.

A first example

So I started to create a few types to run the civilization simulator in Frege:

 

Running the first example

Screenshot from 2014-09-20 14:32:50

So far Frege seems stable, easy to install and very close to Haskell. Now I want to play more with the Java integration.