Last week I started discussing how I am working on improving my approach to diagrams generation, so that it could become acceptable for a Software Engineer.

Currently I write a textual description of my entity-diagrams and I generate the corresponding images using Erd. Erd is a fantastic tool written in Haskell which parses its own DSL and invoke Graphviz, producing images in several different formats. To use it you need to install Graphviz, the Haskell platform and cabal. This is not always straightforward, so I wanted to create a web interface for this program. This would offer a few benefits:

  1. I could then use Erd from wherever I want, without the need to install the toolchain
  2. I could later add Syntax highlighting and GitHub integration
  3. I would have an excuse good enough to play with Haskell before totally forgetting the few things I have learned

The basic idea is pretty simple: the web application will present an editor and when the user clicks on a button the code is sent to the server. The server process the code and generate an image which is sent back to the browser and displayed in the page. It should be enough for the MVP (Minimum Viable Product: now that the enterprise world has taken my soul I have to use these acronyms, right?).

I wrote this piece of software named Erd web-server and it is available on GitHub: https://github.com/ftomassetti/erd-web-server. In the rest of this post I describe its implementation.

Interfacing with Erd

Erd is a nice program written in Haskell. Now, ideally my project would just add Erd to the dependencies and everything should be easy and fine. Unfortunately it is not possible because Erd is a cabal application module (Cabal is the package manager for Haskell). It means that the module cannot be used as a dependency for other modules. So ideally I would like Erd to expose a library and, as a separate project, a console utility wrapping that library. I opened an issue in the Erd project and I hope to able to send a pull-request in the future. For now I just copied the code from Erd in my project. Basically I removed the Main function and I changed one single method (loadER) to accept a Text instance instead of a file handle. In this way I can receive the code in an http request and process it in memory, without the necessity of dumping it to a file.

-- Erd version: loadER takes an Handle
loadER :: String -> Handle -> IO (Either String ER)
loadER fpath f = do
  s <- hGetContents f
  case parse (do { (opts, ast) <- document; return $ toER opts ast}) fpath s of
    Left err -> return $ Left $ show err
    Right err@(Left _) -> return err
    Right (Right er) -> return $ Right er

-- version used by the web server: 
-- loadERFromText process a Text and loadER read from a file
-- and then delegate to loadERFromText
loadER :: String -> Handle -> IO (Either String ER)
loadER fpath f = do
  s <- hGetContents f
  loadERFromText fpath s

loadERFromText :: String -> Text -> IO (Either String ER)
loadERFromText fpath s = do
  case parse (do { (opts, ast) <- document; return $ toER opts ast}) fpath s of
    Left err -> return $ Left $ show err
    Right err@(Left _) -> return err
    Right (Right er) -> return $ Right er

Overview of the Available Routes

The Erd web server just presents a web page where users can edit the description of the ER. Then the user can edit a button and that cause a post request to be sent at /generate with the code of the ER. If things go well an image is generated and it can be retrieved under /generated.

In addition to that we need to serve also assets (Javascript and CSS)

site :: Snap ()
site =
    ifTop viewIndex <|>
    route [ ("generate", generate)
          ] <|>
    dir "assets" (serveDirectory "assets") <|>
    dir "generated" (serveDirectory "generated")

The viewIndex and generate Functions

Serving the index is easy, we just read an HTML and we return it. In the future we could use some template system if we need to have some variable value.

The generate function contains the most interesting part of the code. It operates in the Snap monad from which we can access the request and prepare the response. We have to map IO operations using liftIO which basically translate IO X values in Snap X values.

As first thing we get the body of the request (getRequestBody) and we pass it to processErCode. processErCode could either succeed (if the ER code is correct) or fail:

  • if processErCode fails it return a Left String value, containing a description of the error. In this case we produce a response containing a JSON object with the field error, where we insert the error message
  •  if processErCode succeeds it return a Right ByteString value, containing the bytes of the actual image generated. Ideally we would like to return those bytes in the answer, for the MVP we just dump them to a file which we save locally and we return the name of the file. Note that we generate a random name for the file. This is not ideal and it require to clean up the directory of the generated files from time to time. It is also possible that someone overrides your file accidentally. In theory this solution is very, very poor. In practice it is super simple and it works well enough for my use case :)
processErCode :: String -> IO (Either String ByteString)
processErCode code = do
    -- the name we pass to loadERFromText does not really matter
    res :: Either String ER <- loadERFromText "generated_image.png" (L.pack code)
    case res of
        Left err -> do return $ Left err
        Right er -> do
            let dotted :: G.DotGraph L.Text = dotER er
            let getData handle = do bytes <- BS.hGetContents handle
                                    return bytes
            let fmt :: GraphvizOutput = Png
            gvizRes :: ByteString <- graphvizWithHandle Dot dotted fmt getData
            return $ Right gvizRes

generate :: Snap ()
generate = do
    lContent :: BL.ByteString <- getRequestBody
    let erCode :: String = BS.unpack $ toStrictBS lContent
    liftIO $ putStrLn "(Processing generate request)"
    res <- liftIO $ processErCode $ erCode
    case res of Left errorMsg -> writeBS $ BS.pack $ "{ "error" : "" ++ (escape errorMsg) ++ "" }"
                Right image -> do
                    randomId :: Int <- liftIO $ randomIO
                    let fileName = "generated/diagram_" ++ (show randomId) ++ ".png"
                    liftIO $ BS.writeFile fileName image
                    writeBS $ BS.pack  $ "{ "image" : "" ++ fileName ++ "" }"
                    return ()
    liftIO $ putStrLn "  Done."

toStrictBS = BS.concat . BL.toChunks

escape [] = []
escape ('"':s) = """ ++ escape(s)
escape (c:s) = [c] ++ escape(s)

Serving Assets and Generated Diagrams

We just serve the contents of two directories:

  • assets here we store javascript and CSS files
  • generated this directory contains the diagrams generated at each POST request on /generate

Javascript Code

Finally we need to some work on the client side, basically sending a post request to the server and process the result. It means either:

  • showing the error message in the status box
  • loading the image from the given URL

For implementing the call to the server I have used the promise.js library.

    <script src="assets/promise.min.js"></script>
    <script type="text/javascript">

      window.onload = function() {
        var a = document.getElementById("generateLink");
        a.onclick = function() {
           promise.post('/generate', editor.getValue()).then(function(error, text, xhr) {
               if (error) {
                   console.log('Error ' + xhr.status);
                   return;
               }
               var data = JSON.parse(text);
               if (data.image) {
                  document.getElementById("generatedImage").src = window.location.href + data.image;
                  document.getElementById("status").innerHTML = "Success!";
               } else {
                  document.getElementById("status").innerHTML = "ERROR: " + data.error;
               }
           });

          return false;
        }
      }
    </script>

Screenshots

And now it is show time! From the following image you can notice two things:

  1. the application has a very simple interface: just the editor, the _Generate Diagram_ button, the generated image and a status label saying “Success!” in this case
  2. I am not the most fancy designer out there :)

 Screenshot from 2015-04-19 08:58:13

Final Thoughts

There are many ways we could improve this application:

  • Update the image automatically when the user does not type for a few seconds
  • Report errors inline
  • Syntax highlighting and auto-completion for the editor

On the other hand I am really satisfied with the current result: with few lines of Haskell and Javascript we created an application that help me when working on those fancy diagrams, giving me more time to think about the concepts I want to represent and spending less time remembering the options to generate the diagram or installing tools.