Informations and tutorials on the language workbench Jetbrains MPS

Interview to Vaclav Pech on Jetbrains MPS: the community and the future

If you are reading this blog regularly you know that I am interested in many things but language engineering is the topic closest to my heart. You could have also noticed that I am spending a lot of time working with MPS. If you are instead a new reader you may not know that MPS is an advanced Language Workbench based on projectional editors. In practice you can use it to create Domain Specific Languages with their editors and supporting tools.

A couple of weeks ago I decided to contact Vaclav Pech with a lot of questions about MPS: I love it and there are so many things I wanted to ask about. I had in this way the chance to satisfy my curiosity and take a glimpse at the future of MPS. Vaclav has been so extremely kind and helpful providing detailed and honest answers to all of my questions. It is now a great pleasure to share all of this with other persons interested in MPS.

mps

Introduction

Can you tell us something about yourself?

Hello Federico. First of all, thank you for giving me this wonderful opportunity to tell your readers more about MPS, I’ll be happy to answer all your questions.

I’m a software engineer since 1997, majoring in server-side Java, databases and distributed systems. In the distant past I’ve been working as a developer and consultant on several projects in the finance sector. After joining JetBrains I’ve gradually expanded into the domain of development tools, code analysis, programming languages and DSLs.

What is your role in Jetbrains?

I’ve been with JetBrains since 2003. During these 13 years I’ve had the chance to try several exciting roles. I started in a purely development role for a few years. About four years into it the marketing department won me over and I’ve become a full-time JetBrains evangelist. Then, in about 2011 I jumped back into a more technical role in the MPS team, which, however, included not only in programming tasks, but also a variety of non-programming activities. You can sometimes see me present at conferences, give lectures at universities, deliver MPS trainings, visit partners or shoot technical screen-casts and write documentation.

Adopting MPS

Looking at the forum or the questions on Stack Overflow it seems to me that the interest in MPS is growing. What do you think? What are your impressions on MPS adoption?

Yes, I have the same feeling, and it is not only from the questions, but also the answers that people from the community provide – from their increasing sophistication you see the growing experience among the users.

Our statistics on MPS usage and downloads support the feeling – the active user-base has been growing steadily during the recent years, so that It tripled over the last five years. We’re also happy to have discovered universities using MPS in their curriculum to teach DSLs.

Just like many other open-source projects we live somewhat in the dark not knowing who and how uses our tool. That’s why the Partnership form has been created – to get some feedback from the users on how they use MPS. The discussions that we hold regularly with some of our customers and the feedback that they give us help us direct our focus.

Federico: I agree on the increased level of sophistication. I think that the community is starting to think about best-practices and patterns start to emerge. It is not just about “how do I use feature X” but the community is starting to ask “what is the best way to design the solution to this kind of problem”. We as a community are understanding these things and I find it very exciting.

I saw JetBrains started offering MPS training: how is it going? Any particular plan on that front?

We did, indeed! The team had been receiving requests from customers for quite some time before we decided to enter that field. You can currently choose from two levels of trainings – Introductory and Intermediate, which together cover the fundamentals of language design in MPS. They are typically held at our premises in Prague. We’re also able to do on-site trainings for our customers.

What do you think are the main factors preventing wider adoption of MPS?

I tend to think that this is related to the low adoption of language workbenches in general. In my opinion the experts in domains that would benefit most are not yet aware of the true potential of these modern tools. To speak more concretely about MPS, JetBrains has over time gained pretty good reputation among programmers thanks to its IDE’s, like IntelliJ IDEA or ReSharper. This success story is, however, only marginally relevant for the target audience of MPS. In a sense, marketing of MPS does not overlap with the marketing of the other JetBrains products much.

Federico: I agree that the lack of awareness about DSLs is a major issue. In many fields DSLs could improve dramatically the productivity but people who could benefit from them have simply no idea this is possible. It is also true that marketing MPS is different than marketing IntelliJ IDEA, however I am seeing people becoming more confident in adopting MPS because Jetbrains is becoming more and more well-known and established. This reassures prospects that the company can afford the development of MPS in the long run.

MPS Community

Where are you seeing more interest in MPS? Which countries? Which fields?

I would avoid any generalization until the numbers get considerably larger. The sample we currently have is unfortunately too small and varied to make any sensible conclusion or find a pattern, I’m afraid.

We have users in the insurance business, a tax-collecting government organization, TV studios, research laboratories and, thanks to mbeddr, also numerous embedded software houses.

When in comes to geographical  distribution, we can see Germany at the head of the ploton at the moment, followed closely by a few other European countries. USA, India, China and Israel are also notable in our statistics.

At which conferences or which venues is easier to find MPS enthusiasts?

This is something we yet have to discover. We used to attend CodeGeneration in the past, where we would find many like-minded souls. This conference, however, has been discontinued. At the general developer conferences you always find a few people interested in model-driven development or DSLs, but the critical mass is missing there.

Federico: Perhaps we could think about organizing a small online conference to compare our experiences with MPS. I would love to see how others are using MPS at this moment.

What do you think could help growing a community?

JetBrains has always followed the strategy to win customers by focusing on delivering superior tools – listen carefully to the passionate members of the community and then do your best to satisfy well their needs with the IDEs.

As for MPS, we would like to make it more suitable as a tool that domain experts could use to write DSL code. The tool should offer more guidance through the inherent complexity, which would primarily require numerous improvements in the usability of the user interface. Having a web-based UI would certainly help, as well.

Are you aware of anyone doing interesting things with MPS?

I don’t think there can be a boring project based on MPS. In fact, most of MPS-based projects that I’m aware of do very interesting things. We partner with a government institution, for example, that attempts to model parts of the country’s legislation in MPS. They take MPS to its limits by heavily focusing on tabular notations, in-editor interpretation of the models and seamless integration with several inherited legacy systems.

Fabian Campagne has created MetaR – an MPS-based IDE for statistical data analysis. The Faculty of Mathematics and Physics of Charles University in Prague investigates the paths to automatically create language definitions in MPS for existing languages by importing their grammar. Obviously, there are many more projects that would be worth mentioning. Most of those that gave us their permission are listed on the MPS project web-page.

Federico: I would love to hear more about all these projects. In particular what they are doing at the Charles University sounds pretty amazing. I played a little bit on integrating ANTLR and MPS and I think that Marco Lombardo could have done something in the area too.

Have you ever seen a project built on MPS that surprised you?

The mbeddr project has always been very surprising. These folks not only have created a powerful IDE for embedded software developers to program motor controls, robots, avionics and other embedded devices, but along the way they have built a set of super-handy micro-tools and languages called mbeddr platform that they have generously shared with the community. I’ve seen these gadgets being used in many other MPS projects.

Federico: I am most definitely among their users!

It seems to me that the Mbeddr platform is contributing a lot of interesting stuff. Could we see some of this stuff integrated in MPS in the future?

We’re very grateful to the mbeddr team for packaging and publishing the handy tools and languages that they themselves found useful on their projects. Since the mbeddr platform is open-source, anyone can instantly benefit from it. To make this easy for the users, we’ve exposed the MPS build server to the public and have the mbeddr platform continuously built there, alongside MPS. As the next step we’re considering making a joint distribution of MPS with the mbeddr tools.

We may also be integrating some of the most useful elements directly into MPS, but being busy evolving the core functionality of MPS we will hardly ever catch up with the pace, at which these guys innovate the mbeddr platform.

Federico: I am personally using a lot of stuff from the mbeddr platform, which I think of as a sort of “MPS++”. This makes me desire more and more a way to specify dependencies between languages: I would like to be able to specify declaratively a dependency on a specific version of the mbeddr platform. This leads to my next question on build systems.

One of the thing I am looking forward to is integration with build systems. I saw the work of Sergej Koščejev on a Maven plugin: do you think it will be soon production ready?

Sergej’s initiative received very positive reactions from the community. We will release a prototype as soon as it reaches reasonable maturity. In the first version we will focus on very raw integration with Maven and cover the scenarios of DSL usage in maven-built projects. Later, we will provide the possibility to place MPS languages into maven during the build.

Federico: It is amazing to hear this. I think this component can make a huge difference for our community. I am looking forward to this one.

Future

What are the plans for MPS in the future? Is the public roadmap up to date?

The primary goal for us at the moment is to make MPS more flexible – we would like to better support non-traditional approaches to language development. Expanding on the idea of Language-oriented programming, users should be able to define their own language aspects or modify the existing ones. If, for example, the normal way editors are defined in MPS doesn’t suit you, you should be able to come up with an alternative approach, use it and perhaps also share it with others. There are many roadblocks that we have to clear away, but ultimately MPS should become a truly universal platform.

Another big task is to reorganize the whole code generation process. We would like to detach generators from languages and give the language authors more control over the whole generation scenario.

We would also like to improve the usability in the area of editor definition – provide more intuitive languages, offer better validation of user code, automate many aspects of the dynamic editor behavior (aka side-transforms), as well as text-gen and parser creation.

The notation for editor definition is another area, where we can improve usability a lot. We’re experimenting with notations that would offer more automation

Last but not the least we want to embrace useful visual tools and gadgets that make the editor easier to use for DSL programmers. DSL programmers (aka domain experts) have different skill sets, habits and preferences than professional programmers – they prefer more rigid syntaxes with strong guidance by the IDE, enjoy rich visual assistance, demand explanatory and syntax-aware error messages. The mbeddr project has gone a long way towards providing intuitive visual assistance and we would like to follow their footsteps in this.

Following the previous question: will we see MPS on the web? Is the JetPad project a priority?

You can actually already see it. The Census Analyser (https://censusanalyzer.com/) project is an experimental implementation of a web-based projectional editor (leveraging the JetPad library). At the moment, we’re not trying to enable web-deployment of languages designed in MPS. This will come later, once we gain some more experience in the domain. Currently, we experiment with a slightly modified and simplified approach to language definition for the purpose of web-deployment. The language designer uses a set of lightweight MPS-based languages to define her languages in IntelliJ IDEA. The generated Java/JetPad code has then no reliance on MPS, it can be deployed on a web-server and enabled to assist editing code in the browser. Through Java and JavaScript API it can communicate with the rest of the web application. Don’t forget that we are still at an early stage of development, so the distribution and business model of this tool hasn’t been decided yet.

Federico: This is so incredibly exciting. Definitely you like to do cool stuff at Jetbrains. I think that all language engineers will be excited to see this project and they will follow closely its evolution: I most definitely will.

Do you plan to develop a GPL specifically designed to take advantage of MPS?

We do not have such plans and it is not even clear, whether such a GPL can be created. BaseLanguage is the default GPL that is being used across MPS. It was originally meant to be independent from Java, but the need to generate Java from BaseLanguage influenced its design heavily, so, in the end, it became almost identical to Java and thus exhibits very little syntax alignment towards projectional editing. We might possibly try to build a platform-agnostic language on top of BaseLanguage in the future, but, on the other hand, implementing multiple projectional GPLs as copies of existing GPLs (JavaScript, C#, etc.) to ease porting DSLs to other platforms currently looks like a more viable path to take. Let me expand a bit on this topic.

Despite all the efforts that JetBrains has put into making the usability of the MPS editor as text-like as possible and despite the close similarity between MPS and say, IntelliJ IDEA, when it comes to structural manipulation of code, developers using GPLs still tend to struggle with projectional editing. They miss the possibility to (temporarily) treat their code as pure text in order to manipulate it on the character granularity. Since the main advantages of projectional editing, such as language composition and non-parseable notations, are not critical for GPLs as much as for DSLs, the balance between pros and cons favors textual GPL languages.

The syntax of a hypothetical projectional GPL would, in my opinion, have to lower the need for direct character manipulation and increase the leverage of structured, possibly non-parseable (graphical) notations. Since the current GPLs are well served with the traditional text-based IDEs, we see more potential for MPS in the DSL arena.

Federico: I agree that it is easier to see the benefits of MPS for DSLs. After all domain experts have today very limited tools to formalize their knowledge and automatically process it. Developers have instead advanced languages and great tools (like IntelliJ IDEA): they have learnt how to use such languages and going around their limitations so I think they are less prone to see the benefits of projectional editing. I still think it would be very interesting to use MPS to create extensible, more flexible GPLs but we have to figure out how to take advantage of the possibilities of MPS. Right now we are too used to textual languages and that limits our creativity.

Sometimes people see that Jetbrains is betting on Kotlin and MPS and they wonder about this split of focus. I know that Jetbrains is large enough to support both these efforts but would you share some insights on this? Do you see any relation between the two projects?

You have to keep in mind that Kotlin serves a different purpose than MPS. Kotlin offers an alternative to the Java language and fights for the hearts and souls of Java programmers. MPS is a DSL tool, which targets primarily the areas outside of the mainstream programming and the domains outside of the reach of languages and tools like Kotlin and IntelliJ IDEA, respectively. We’re trying to address domain experts, who are not necessarily full-time professional programmers and who live in and enjoy fields as varied as electrical engineering, insurance, software modeling, biology, data analysis, legislation, etc. Kotlin, or Java for that matter, are usually too generic to support their use cases nicely.

Is it true that YouTrack 2 will be developed in Kotlin instead of MPS? Is that a bad sign for MPS?

Yes, the domain of web application development offers such a large number of great technologies and tools for Java developers that it was impossible for the MPS-based set of web languages, which the YouTrack team had created and used, to keep pace. The YouTrack team made a sensible decision to rely on proven web technologies, instead of maintaining their own ones, in order to make their development more efficient.

This only confirms that the sweetspot for MPS lies in domains outside of the programming mainstream, which cannot be well served with the mainstream technology stack.

Conclusions

I think everyone listening to your talks or screencast will be thinking the same question: how did you manage to get such a wonderful English accent?

Thank you 🙂 It must have been through osmosis. I should perhaps thank all my British friends and colleagues for influencing me.

Anything else do you want to add?

Thank you for all the questions. I really appreciate your help in spreading the good word on language engineering.

Federico: it was a great pleasure for me! I have been interested in MPS for several years and having the possibility to ask questions to Vaclav has been amazing for me. I hope other MPS enthusiasts can also share my enthusiasm.

You may be also interested in:

ANTLR and Jetbrains MPS: Parsing files and display the AST using the tree notation

Itemis did it again: they just released a new very cool plugin for Jetbrains MPS. This one permits to define new tree editors.

They look like this:

ast

In this post we are going to see:

  • how to use ANTLR parsers inside MPS
  • how to represent the parsed AST using the tree notation

In particular we are going to use the ANTLR grammar which parses… ANTLR grammars. How meta is that? The very same approach could be used for every ANTLR grammar, of course.

Also always code is is available on GitHub.

Dependencies

First of all you need to install Jetbrains MPS. Grab your free copy here.

To use the tree notations you should install the mbeddr platform. Just go here, download a zip and unzip it among the plugins of your MPS installation.

All set, time to do some programming.

Packaging ANTLR to be used inside MPS

In a previous post we discussed how to use an existing ANTLR grammar in Java projects using Gradle. We will apply that technique also here.

We start by download the grammar from here: https://github.com/antlr/grammars-v4/tree/master/antlr4

We just do some minor changes by including directly LexBasic into ANTLRv4Lexer. Note that we need also the LexerAdaptor.

For simplifying the usage we create a Facade:

Now we need a build file:

You may want to run:

  • gradle idea to create a Jetbrains IDEA project
  • gradle fatJar to create a Jar which will contain our compiled code and all the dependencies

Good. Now to use this parser into MPS we start by creating a project. In the wizard we select also the runtime and sandbox options. Once we have done that we should copy our fat jar under the models directory of the runtime solution. In my case I run from the directory of the Java project this command:

Now we need to make MPS aware of that Jar. Lets’s select the sandbox solution and first add the jar to the models:

window1

Then we add it also to the libraries:

window2

Now the content of the JAR should appear among the stubs of the runtime solution.

stubs

 

Creating MPS nodes from AST nodes

Now we are going to build a new concept named AntlrImporter. We will use it to select and import ANTLR grammars into MPS:

importer

The Concept structure will be pretty simple:

AntlrImporter

We need also concepts for the AST nodes we are going to import. First of all, we will define the abstract concept AstNode. Then we will define two subconcepts for the terminal and non-terminal AST nodes.

terminal_nodenon_terminal_node

Now let’s take a look at the editor for the AntlrImporter.

antlrimporter_editor

The first swing component is a button which opens a file chooser. In this way, we can easily select a file and set the property path. Or we can edit it manually if we prefer.

 

file_chooser

Once we have selected a File we can import it by clicking on the second button

button

The import logic is in importModel, a method in the behavior of AntlrImporter.

import_logic

Good. That is it. With that we can parse any ANTLR grammar and get it into MPS. Now we have just to use a nice representation. We are going for the tree notation.

Using the tree notation

The tree notation is surprising easily to use.

Let’s start by adding com.mbeddr.mpsutil.treenotation.styles.editor to the dependencies of the editor aspect of our language.

editor_deps

We will need also the com.mbeddr.mpsutil.treenotation to be among the used languages.

editor_langs

The editor for NonTerminalNode consists of a single tree cell. The top part of the tree cell represents this node. We will use the ruleName to represent it. In the bottom part instead we should pick the relation contains the children to be displayed in the tree

non_terminal_editor

We can put the cursor on the tree drawing between the top and the bottom part (the “/|\” symbol) and open the inspector. There we can use style attributes to customize the appearance of the tree

non_terminal_inspector

We just decide to show the tree from left-to-right instead that top down. Then we decide to add more spaces between the parent and the children when there are too many children. In this way the lines to not overlap too much.

This is how it looks without the property

crowded

This is how it looks with the property set

not_crowded

There are other properties that can be used to control the color and the thickness of the lines, for example. Or you could add shapes at the extremes of the lines. For now we do not need these features, but it is nice to know they are there.

The editor for TerminalNode is very simple

terminal_editor

Conclusions

Over the years MPS became more stable and easier to use. It has reached the point at which you can be very productive using it. Projectional editing is an idea that has been around for a while and there are other implementations available like the Whole Platform. However MPS has reached a very high level of maturity.

What I think we still miss are:

  • processes and best practices: how should we manage dependencies with other MPS projects? How should we integrate with Java libraries?
  • examples: there are surprisingly few applications which are publicly available. After all, many users develop DSLs for their specific usages and do not intend to share them. However, this means we have few opportunities to learn from each other
  • extensions: the Mbeddr team is doing an amazing job providing a lot of goodies as part of the Mbeddr platform. However, they seem the only ones producing reusable components and sharing them

I think this is now time to understand together what we can achieve with projectional editing. In my opinion these are going to be very interesting times.

If I have to express one wish is that I would like to hear more about how others are using MPS. If you are out there, please knock. And leave a comment 🙂

mps-bytecode: creating, loading, modifying, saving and executing JVM class files using Jetbrains MPS

I recently created this project to edit JVM class-files inside Jetbrains MPS.

Why?

  • It was fun
  • It is a great tool to learn about the JVM internals. Do you know what a frame is? How dynamic invoke works?
  • Instead of generating Java code for my applications I could translate to mps-bytecode. More about this later.

Code is on GitHub, as always: mps-bytecode

Opening an existing class files

Consider this simple program:

You can compile it simply by running:

Now you have a class file. You can import that class file in MPS. To do that you just create a ClassFileLoader, type the filename of the class file and click on load

ex1.mkv

I will add support to import whole JARs. It should be pretty simple.

Taking a look at the class file

One key element of the class file is the constant pool. It is basically an indexed list of elements which define constants. Each of these elements has one type and it is followed by some data.

Look at this example:

  • We have several UTF8 elements (e.g., 7, 8, 10, 12, 12, 13, 14, etc.).
  • We have also Strings, which just contain a pointer to an UTF8 element.

For example the element 3 is composed by one byte indicating the type of the element (Constant Pool String) and one byte indicating the index of the UTF8 string (in this case 18, corresponding to the string “Hello world!”).

Screenshot from 2016-04-24 11-29-17

Note that when we loaded the file we created proper references between class pool elements. That means that if you move the referenced element (e.g., 18), the element holding the reference (e.g., 3) will be updated automatically:

ex2.mkv

We have then fields and methods. This class contains two methods, even if we defined just one. The additional method is the default constructor:

Screenshot from 2016-04-24 11-40-24

It contains the access flags, the name and descriptor indexes (the actual strings “<init>” and “()V” are contained in the Constant Pool). After that there are the attributes. One very important attribute is the “Code” attribute which contains the instructions which are executed by the JVM when this method is invoked.

In addition to that there is one attribute of the code attribute: it basically tells us the line numbers corresponding to the different instructions

This is instead our main method:

Screenshot from 2016-04-24 11-40-37

Not much more code here:

It does the following:

  • take the static field specified at index 2 (the index is 2 0 where 0 is the high byte and 0 the low byte -> 2 in total). It is the out field of System
  • it pushes on the stack the String constant at index 3 (Hello world!)
  • it invokes the method specified at index 4 (println). The method will take the parameters from the stack
  • it returns

 

Running a class file from inside MPS

You can also run classes inside MPS.  You have to define Executors like this one:

Screenshot from 2016-04-24 11-54-09

The executor defines the class to invoke, the classpath to use and the arguments passed to the main function.

In this case we just specify a link to a class file, no additional classpath entries and no arguments

ex3

 

Modifying and saving a class file

We have seen how to import a Class File but you can also create them from scratch. You can then edit the class files as you want and save them simply by generating the project containing them. The editor knows about the semantic of class files so it will help you writing consistent class files. For example, if an instruction requires using an index to a Constant Pool String element this will be verified for you. Writing class files is not trivial, so a little help is not bad.

What can this be used for?

Aside from having fun and learning more about the JVM, my goal is to define new languages to compile for the JVM, without having to generate Java source code.

Now, in many cases it can be simpler to just generate Java and let the Java compiler handle the bytecode generation for you. However on one side there is possible valid “JVM code” which cannot be obtained by compiling Java. This could involve invoke dynamic, for example. In addition to that there is a guts-feeling factor: if I generate Java code and compile that my language does not feel completely to be a real language. I know it is not rationale and there are plenty of proper languages which generated C code instead of producing machine code, however this is a sensation I cannot fight for now. So I will play a bit more with these concepts and see what I learn from this.

Also, mps-bytecode could also be used to examine JVM classes obtained from several languages.

What do you think of this? Any other usage you can envision for it?

 

Tabular projections in Jetbrains MPS: let’s start building an accounting system

Jetbrains MPS is an amazingly flexible tool which permits to use several kinds of projections. Today we are going to take a look at tables.

What is Jetbrains MPS to me

Jetbrains MPS is a Language Workbench: a system to design languages and supporting tools.

This is one definition of MPS, but for me it is also something different. I think of it as Lisp where you can customize also tool support, instead of being confined to parenthesis. It is a tool to write very easily metamodels and editors which I can evolve and extend. If I need it I can insert code I do whatever I want. I guess it is for programmers what Excel can be for other human-beings: a very flexible tool that give you the power to play with your ideas.

What are we going to build?

So, after a few months that I started my consulting activities business is going well. Which is very nice but it also means that I am getting a bit tired of doing all the accounting: writing invoices, tracking expenses. Sure, my business consultant handle the tax part but still I need to ensure I deliver to him all the right documents. In addition to that I think it is healthy to keep an eye on your finances, right?

So I started to build a simple solution, customized for my needs. Something that I could evolve over time as my business evolves and my needs change, leading to changes in the processes I follow.

I started building a template system to generate the invoice using the Google Drive API and the Google Execution Service (presented in this post). After that I needed something to collect the data of my invoices and track the expenses.

Tables, my kingdom for some tables

We are going to use a plugin named mps-tables.  Mps-tables has been written by Sascha Lißon and it is used in the mbeddr platform. You can think of the mbeddr platform as a sort of “MPS++”: a collection of nice extensions to simplify your life as language designer.

This is how I would edit my invoices:

output

  • The number of the invoice and date could be automatically filled in (and I could change them if I want)
  • The emitter would be me, with the possibility to change it
  • I could select the client from the clients I defined in another file
  • I would just add lines with a description, an amount and a VAT rate
  • The VAT amount and the totals would be calculated

Now, I could want to add later some support to calculate the VAT automatically considering a few rules. In particular, I am a French company and most of my clients are companies outside France so I have not to apply VAT to them. That means that the VAT Rate is 0% in most cases. I have instead to apply a VAT of 20% to French companies and private customers. In practice the VAT rate could be determined looking at the customer.

Ok, how we build this?

All the magic happens in the editor:

Screenshot from 2016-04-16 09-56-34

We start by adding a cell of type de.slisson.mps.tables.structure.table. Then we insert a vertical element inside the table. In the header part of the vertical element (the one starting with “c<” and terminated by “>”) we just list the titles of the columns: Description, Amount, VAT Rate, VAT, Total. Contained in this vertical element we have the horizontal element. Here in the header, we specify the line-numbers of our invoice. We basically say that:

  • We have just to display the line number as the index + 1 (indexes start from 0)
  • We can insert new lines
  • We can remove new lines until we have just one line left. That one cannot be removed

Screenshot from 2016-04-16 09-56-54

Time to show the content of each line. Nothing too fancy here: the number of columns is always 5 and we want one line for each line of the invoice (not surprising, eh?). In the first three cells we just put the values for description, amount, and vatRate. The user can edit it freely. The last two columns are instead calculated. They are the VAT amount (amount * vatRate) and the total (amount + VAT amount).

Screenshot from 2016-04-16 09-57-09

Finally, we show the total. Here all the cells are calculated. Note also the custom cell just before the query element. It is a hack to show an horizontal line as long as the table.

This is just a simple example of using MPS to build a simple application to collect data. This is not the “official” goal of MPS. MPS is intended to build languages and language extensions. However it has:

  • the power to define a metamodel very easy,
  • the possibility to mix several projections (we have seen text and tables, but there is more),
  • the freedom to easily integrate logic wherever I need it

All of these features MPS very convenient to build applications to manipulate data, validate it, and process it for any goal. I think this make MPS somehow usable in place of frameworks like EMF Parsley (however Parsley has the big advantage of producing both desktop and web applications). I want to think more about this. After all it seems that we are just starting to understand the power of MPS.

Code is available on GitHub:
https://github.com/ftomassetti/MpsAccounting

Update

After I published the code I received a PR from the author of mps-tables. He kindly suggested a much cleaner way to obtain the same result.

Basically in the table we can simply says that we are going to iterate on the invoice lines. We just specify how to represent the headers. How to represent the single lines is defined in the editor for the InvoiceLine itself, by using a partial table.

Selection_004

We just specify how to represent the headers. How to represent the single lines is defined in the editor for the InvoiceLine itself, by using a partial table.

Selection_005

I really think that the potentiality of mps-tables and the rest of the extensions available in the mbeddr platform are amazing and yet to be understood by many MPS users. My little goal is try to share some of their potentialities with the rest of the community.

Raising the level of abstraction: what if we tried to do that bottom up?

It is clear that software development is going in the direction of raising the level of abstraction since it was born.

The reason is simple: we can juggle with so many concepts in our mind. If we are busy thinking about what we stored in the EAX register it is difficult to think about the bigger picture. As we raise the level of abstraction we start using bigger bricks and we can see farther.  Once we think in terms of objects the OOP design patterns can emerge, because they build on combinations of a few concepts at our current level of abstraction.

How are we moving to higher levels of abstractions? By hiding the details. We hide how we use registers, how we allocate memory, how pointers are handled, how polymorphism is implemented.

There are several way we can achieve that: normally we build new languages from which we generate or compile to some lower level abstraction. For example many C++ compilers at the beginning were just spitting out some C. Also compilers for Eiffel were using the same strategy. One thing that instead was not tried was to evolve languages adding new concepts as they emerge. In this case I am thinking about a gradual process which would target micro-patterns. Things like getters or setters in Java. Or things like this one:

What we are doing here? we are just returning self.data[code]. If it was not initialized we would set it to a default value ({}). Ok, lazy initialization: this is a concept common and comprehensible per se. What if we could add this concept and see if it is then used to build up new patterns?

This would be trivial to do using a tool like Jetbrains MPS.

I plan to play a little bit with this idea in next week and see where it leads me.

Getting started with Jetbrains MPS: how to define a simple language (screencast)

I am blessed (or cursed) with having many different interests in software development. However my deepest love is with language engineering. There are different tools and techniques but Jetbrains MPS is particularly fascinating.

To help people getting started or just get the feeling of how it is to develop with Jetbrains MPS I recorded this short screencast.

 

 

If you are not familiar with this tool here there are the basic ideas.

Jetbrains MPS is a projectional editor: that means that it internally stores the information contained in the different nodes.

If you create a model, then change how certain nodes should look, and you reopen the model it will be automatically using the new “look & feel” that you just. This is simply because what is stored is the pure information, while the appearance is recreated each time you open a model. In other words when you open a model it is projected in a form that you defined and that should be easy to understand for a human being.

This is very different from what happens with text language: suppose you create your language and then decide to replace braces with brackets. You have now to manually go over all your existing source files and modify them, because in that case what is saved is not the pure information, the abstract syntax tree, but its representation: the textual form.

In Jetbrains MPS you work on the different concerns of your language separately: you first define the single concept and the information they contain, then you work on the other aspects. There are several like typesystem, generation, dataflow, etc. In this short screencast we just look at the concept definition and the editor aspect.

One of the advantages of a projectional editor is that you can change the appearance of your nodes without invalidating your existing models. That is great to incrementally improve the appearance of the single nodes and the way they can be edited. In the screencast we start with the default representation of the concept we defined, as provided by default by Jetbrains MPS. Then we refine the editors in a couple of steps until we have a representation which is good enough.

At the end of the screencast we have a language we can use to define very simple todo lists. The possibility to improve it are endless:

  • we could add more concepts: like priority, deadlines and tags for each single task
  • we could add the possibility to mark a task as done and add a nice shortcut to do so
  • we could integrate the language with some other tool: what if we could synchronize our models with Trello or with workflowy?
  • we could support subtasks

Question: have you ever thought of creating your own language, either a general purpose language or a domain specific language? If so did you actually get started?

 

The C to mbeddr-C importer: a first look

Currently I am in Munich, working at Fortiss, an institute of the Technische Universität München. I am here to work on mbeddr, which is a super-cool project.

mbeddr supports embedded software development based on an extensible
version of C language and IDE. Existing extensions include interfaces with pre- and postconditions, components, state machines and physical units, as well as support for requirements tracing and product line variability. Based on these abstractions, mbeddr also supports formal verification based on model checking and SMT solving.

What it means in practice?

We are using a Jetbrains MPS to build an extensible variant of C. Using mbeddr the user could write plain C code and use, when it is needed, some extra features.

One example? State machines. Instead of manually encoding them using a bunch of switch the user could use specific constructs. Some advantages:

  • it is less error prone,
  • it is faster,
  • the editor understand what you are doing and catch your errors,
  • you can verify that some properties holds (e.g., are all the states reachable?)

In other words mbeddr permits to work to higher levels of abstractions.

How C and mbeddr-C differs

Mbeddr supports almost all the features of C. It does not support error-prone features like the preprocessor. For most of the usages of the preprocessor mbeddr offers higher-level constructs. Consider for example variability. In C you would see this code:

In mbeddr-C you would write something like:

Variability in mbeddr

 

in practice you have the possibility to decorate statements and functions with their presence condition. This is cool for many reasons, first of all the possibility for mbeddr to verify that your code is valid under every possible configuration.

What is my contribution?

I am building the importer which converts plain C to mbeddr-C. It interprets how the preprocessor is used in C code and translate it to higher level concepts.

In this video I show the current state of the importer.

 

If you want some more details about the importer you can read this very preliminary paper, accepted for the Reverse Engineering workshop to be held in Genova on the 5th of March.

Screencast – How to show the content of a file in Jetbrains MPS

A few days ago I recorded a short screencast to show how it is possible to visualize a file inside Jetbrains MPS.

Tutorial: how to add an Action to the Jetbrains MetaProgramming-System

The Jetbrains MetaProgrammin-System (MPS)

The MetaProgramming-System is a Language Workbench based on projectional editing and it is the most advanced publicly available tool of this kind.

If you have not yet took a look at it you should, really. It makes possible (and easy!) to evolve and mix programming languages and modelling languages providing specific tool support.

Have you ever dreamed to improve a little bit your favourite language (e.g., Java). Well, with MPS it is just some clicks away. To get familiar with it take a look at the basic tutorials at http://www.jetbrains.com/mps/documentation/index.html.

In this tutorial we will see how to add an action to the IDE.

How to write an action for MPS

The tutorial was tested using MPS 2.0.6

We want to add an action that we can invoke from the “Tools” menu.

The first step is create a new Project (File -> New Project).

Then we should add the plugin aspect to the language.

Now we have to create a Group.

This group will contain our Action. Set its name (ExportGroup) and add an element list where the editor indicates an error. Under the modifications section specify that the editor should go under the Tools group.

Then we should add an action.

Set the name and the caption. To test that our action get invoked we will show a Message using JOptionPane. To do that we should add the stub to our language. Select our language and from the context menu select “Module properties”. In the common tab add under Accessory models the javax.swing stub.

We can now add the code to show a message.

Add the Action to the group.

Now build the language (from the context menu of the Language select “Make Language ‘name of your language’ “). In the Tools menu it is now present our action.

If we invoke it a message appears.

In the next tutorial we will add some code into the action to export the models as XML.