If you are reading this blog regularly you know that I am interested in many things but language engineering is the topic closest to my heart. You could have also noticed that I am spending a lot of time working with MPS. If you are instead a new reader you may not know that MPS is an advanced Language Workbench based on projectional editors. In practice you can use it to create Domain Specific Languages with their editors and supporting tools.

A couple of weeks ago I decided to contact Vaclav Pech with a lot of questions about MPS: I love it and there are so many things I wanted to ask about. I had in this way the chance to satisfy my curiosity and take a glimpse at the future of MPS. Vaclav has been so extremely kind and helpful providing detailed and honest answers to all of my questions. It is now a great pleasure to share all of this with other persons interested in MPS.

mps

Introduction

Can you tell us something about yourself?

Hello Federico. First of all, thank you for giving me this wonderful opportunity to tell your readers more about MPS, I’ll be happy to answer all your questions.

I’m a software engineer since 1997, majoring in server-side Java, databases and distributed systems. In the distant past I’ve been working as a developer and consultant on several projects in the finance sector. After joining JetBrains I’ve gradually expanded into the domain of development tools, code analysis, programming languages and DSLs.

What is your role in Jetbrains?

I’ve been with JetBrains since 2003. During these 13 years I’ve had the chance to try several exciting roles. I started in a purely development role for a few years. About four years into it the marketing department won me over and I’ve become a full-time JetBrains evangelist. Then, in about 2011 I jumped back into a more technical role in the MPS team, which, however, included not only in programming tasks, but also a variety of non-programming activities. You can sometimes see me present at conferences, give lectures at universities, deliver MPS trainings, visit partners or shoot technical screen-casts and write documentation.

Adopting MPS

Looking at the forum or the questions on Stack Overflow it seems to me that the interest in MPS is growing. What do you think? What are your impressions on MPS adoption?

Yes, I have the same feeling, and it is not only from the questions, but also the answers that people from the community provide – from their increasing sophistication you see the growing experience among the users.

Our statistics on MPS usage and downloads support the feeling – the active user-base has been growing steadily during the recent years, so that It tripled over the last five years. We’re also happy to have discovered universities using MPS in their curriculum to teach DSLs.

Just like many other open-source projects we live somewhat in the dark not knowing who and how uses our tool. That’s why the Partnership form has been created – to get some feedback from the users on how they use MPS. The discussions that we hold regularly with some of our customers and the feedback that they give us help us direct our focus.

Federico: I agree on the increased level of sophistication. I think that the community is starting to think about best-practices and patterns start to emerge. It is not just about “how do I use feature X” but the community is starting to ask “what is the best way to design the solution to this kind of problem”. We as a community are understanding these things and I find it very exciting.

I saw JetBrains started offering MPS training: how is it going? Any particular plan on that front?

We did, indeed! The team had been receiving requests from customers for quite some time before we decided to enter that field. You can currently choose from two levels of trainings – Introductory and Intermediate, which together cover the fundamentals of language design in MPS. They are typically held at our premises in Prague. We’re also able to do on-site trainings for our customers.

What do you think are the main factors preventing wider adoption of MPS?

I tend to think that this is related to the low adoption of language workbenches in general. In my opinion the experts in domains that would benefit most are not yet aware of the true potential of these modern tools. To speak more concretely about MPS, JetBrains has over time gained pretty good reputation among programmers thanks to its IDE’s, like IntelliJ IDEA or ReSharper. This success story is, however, only marginally relevant for the target audience of MPS. In a sense, marketing of MPS does not overlap with the marketing of the other JetBrains products much.

Federico: I agree that the lack of awareness about DSLs is a major issue. In many fields DSLs could improve dramatically the productivity but people who could benefit from them have simply no idea this is possible. It is also true that marketing MPS is different than marketing IntelliJ IDEA, however I am seeing people becoming more confident in adopting MPS because Jetbrains is becoming more and more well-known and established. This reassures prospects that the company can afford the development of MPS in the long run.

MPS Community

Where are you seeing more interest in MPS? Which countries? Which fields?

I would avoid any generalization until the numbers get considerably larger. The sample we currently have is unfortunately too small and varied to make any sensible conclusion or find a pattern, I’m afraid.

We have users in the insurance business, a tax-collecting government organization, TV studios, research laboratories and, thanks to mbeddr, also numerous embedded software houses.

When in comes to geographical  distribution, we can see Germany at the head of the ploton at the moment, followed closely by a few other European countries. USA, India, China and Israel are also notable in our statistics.

At which conferences or which venues is easier to find MPS enthusiasts?

This is something we yet have to discover. We used to attend CodeGeneration in the past, where we would find many like-minded souls. This conference, however, has been discontinued. At the general developer conferences you always find a few people interested in model-driven development or DSLs, but the critical mass is missing there.

Federico: Perhaps we could think about organizing a small online conference to compare our experiences with MPS. I would love to see how others are using MPS at this moment.

What do you think could help growing a community?

JetBrains has always followed the strategy to win customers by focusing on delivering superior tools – listen carefully to the passionate members of the community and then do your best to satisfy well their needs with the IDEs.

As for MPS, we would like to make it more suitable as a tool that domain experts could use to write DSL code. The tool should offer more guidance through the inherent complexity, which would primarily require numerous improvements in the usability of the user interface. Having a web-based UI would certainly help, as well.

Are you aware of anyone doing interesting things with MPS?

I don’t think there can be a boring project based on MPS. In fact, most of MPS-based projects that I’m aware of do very interesting things. We partner with a government institution, for example, that attempts to model parts of the country’s legislation in MPS. They take MPS to its limits by heavily focusing on tabular notations, in-editor interpretation of the models and seamless integration with several inherited legacy systems.

Fabian Campagne has created MetaR – an MPS-based IDE for statistical data analysis. The Faculty of Mathematics and Physics of Charles University in Prague investigates the paths to automatically create language definitions in MPS for existing languages by importing their grammar. Obviously, there are many more projects that would be worth mentioning. Most of those that gave us their permission are listed on the MPS project web-page.

Federico: I would love to hear more about all these projects. In particular what they are doing at the Charles University sounds pretty amazing. I played a little bit on integrating ANTLR and MPS and I think that Marco Lombardo could have done something in the area too.

Have you ever seen a project built on MPS that surprised you?

The mbeddr project has always been very surprising. These folks not only have created a powerful IDE for embedded software developers to program motor controls, robots, avionics and other embedded devices, but along the way they have built a set of super-handy micro-tools and languages called mbeddr platform that they have generously shared with the community. I’ve seen these gadgets being used in many other MPS projects.

Federico: I am most definitely among their users!

It seems to me that the Mbeddr platform is contributing a lot of interesting stuff. Could we see some of this stuff integrated in MPS in the future?

We’re very grateful to the mbeddr team for packaging and publishing the handy tools and languages that they themselves found useful on their projects. Since the mbeddr platform is open-source, anyone can instantly benefit from it. To make this easy for the users, we’ve exposed the MPS build server to the public and have the mbeddr platform continuously built there, alongside MPS. As the next step we’re considering making a joint distribution of MPS with the mbeddr tools.

We may also be integrating some of the most useful elements directly into MPS, but being busy evolving the core functionality of MPS we will hardly ever catch up with the pace, at which these guys innovate the mbeddr platform.

Federico: I am personally using a lot of stuff from the mbeddr platform, which I think of as a sort of “MPS++”. This makes me desire more and more a way to specify dependencies between languages: I would like to be able to specify declaratively a dependency on a specific version of the mbeddr platform. This leads to my next question on build systems.

One of the thing I am looking forward to is integration with build systems. I saw the work of Sergej Koščejev on a Maven plugin: do you think it will be soon production ready?

Sergej’s initiative received very positive reactions from the community. We will release a prototype as soon as it reaches reasonable maturity. In the first version we will focus on very raw integration with Maven and cover the scenarios of DSL usage in maven-built projects. Later, we will provide the possibility to place MPS languages into maven during the build.

Federico: It is amazing to hear this. I think this component can make a huge difference for our community. I am looking forward to this one.

Future

What are the plans for MPS in the future? Is the public roadmap up to date?

The primary goal for us at the moment is to make MPS more flexible – we would like to better support non-traditional approaches to language development. Expanding on the idea of Language-oriented programming, users should be able to define their own language aspects or modify the existing ones. If, for example, the normal way editors are defined in MPS doesn’t suit you, you should be able to come up with an alternative approach, use it and perhaps also share it with others. There are many roadblocks that we have to clear away, but ultimately MPS should become a truly universal platform.

Another big task is to reorganize the whole code generation process. We would like to detach generators from languages and give the language authors more control over the whole generation scenario.

We would also like to improve the usability in the area of editor definition – provide more intuitive languages, offer better validation of user code, automate many aspects of the dynamic editor behavior (aka side-transforms), as well as text-gen and parser creation.

The notation for editor definition is another area, where we can improve usability a lot. We’re experimenting with notations that would offer more automation

Last but not the least we want to embrace useful visual tools and gadgets that make the editor easier to use for DSL programmers. DSL programmers (aka domain experts) have different skill sets, habits and preferences than professional programmers – they prefer more rigid syntaxes with strong guidance by the IDE, enjoy rich visual assistance, demand explanatory and syntax-aware error messages. The mbeddr project has gone a long way towards providing intuitive visual assistance and we would like to follow their footsteps in this.

Following the previous question: will we see MPS on the web? Is the JetPad project a priority?

You can actually already see it. The Census Analyser (https://censusanalyzer.com/) project is an experimental implementation of a web-based projectional editor (leveraging the JetPad library). At the moment, we’re not trying to enable web-deployment of languages designed in MPS. This will come later, once we gain some more experience in the domain. Currently, we experiment with a slightly modified and simplified approach to language definition for the purpose of web-deployment. The language designer uses a set of lightweight MPS-based languages to define her languages in IntelliJ IDEA. The generated Java/JetPad code has then no reliance on MPS, it can be deployed on a web-server and enabled to assist editing code in the browser. Through Java and JavaScript API it can communicate with the rest of the web application. Don’t forget that we are still at an early stage of development, so the distribution and business model of this tool hasn’t been decided yet.

Federico: This is so incredibly exciting. Definitely you like to do cool stuff at Jetbrains. I think that all language engineers will be excited to see this project and they will follow closely its evolution: I most definitely will.

Do you plan to develop a GPL specifically designed to take advantage of MPS?

We do not have such plans and it is not even clear, whether such a GPL can be created. BaseLanguage is the default GPL that is being used across MPS. It was originally meant to be independent from Java, but the need to generate Java from BaseLanguage influenced its design heavily, so, in the end, it became almost identical to Java and thus exhibits very little syntax alignment towards projectional editing. We might possibly try to build a platform-agnostic language on top of BaseLanguage in the future, but, on the other hand, implementing multiple projectional GPLs as copies of existing GPLs (JavaScript, C#, etc.) to ease porting DSLs to other platforms currently looks like a more viable path to take. Let me expand a bit on this topic.

Despite all the efforts that JetBrains has put into making the usability of the MPS editor as text-like as possible and despite the close similarity between MPS and say, IntelliJ IDEA, when it comes to structural manipulation of code, developers using GPLs still tend to struggle with projectional editing. They miss the possibility to (temporarily) treat their code as pure text in order to manipulate it on the character granularity. Since the main advantages of projectional editing, such as language composition and non-parseable notations, are not critical for GPLs as much as for DSLs, the balance between pros and cons favors textual GPL languages.

The syntax of a hypothetical projectional GPL would, in my opinion, have to lower the need for direct character manipulation and increase the leverage of structured, possibly non-parseable (graphical) notations. Since the current GPLs are well served with the traditional text-based IDEs, we see more potential for MPS in the DSL arena.

Federico: I agree that it is easier to see the benefits of MPS for DSLs. After all domain experts have today very limited tools to formalize their knowledge and automatically process it. Developers have instead advanced languages and great tools (like IntelliJ IDEA): they have learnt how to use such languages and going around their limitations so I think they are less prone to see the benefits of projectional editing. I still think it would be very interesting to use MPS to create extensible, more flexible GPLs but we have to figure out how to take advantage of the possibilities of MPS. Right now we are too used to textual languages and that limits our creativity.

Sometimes people see that Jetbrains is betting on Kotlin and MPS and they wonder about this split of focus. I know that Jetbrains is large enough to support both these efforts but would you share some insights on this? Do you see any relation between the two projects?

You have to keep in mind that Kotlin serves a different purpose than MPS. Kotlin offers an alternative to the Java language and fights for the hearts and souls of Java programmers. MPS is a DSL tool, which targets primarily the areas outside of the mainstream programming and the domains outside of the reach of languages and tools like Kotlin and IntelliJ IDEA, respectively. We’re trying to address domain experts, who are not necessarily full-time professional programmers and who live in and enjoy fields as varied as electrical engineering, insurance, software modeling, biology, data analysis, legislation, etc. Kotlin, or Java for that matter, are usually too generic to support their use cases nicely.

Is it true that YouTrack 2 will be developed in Kotlin instead of MPS? Is that a bad sign for MPS?

Yes, the domain of web application development offers such a large number of great technologies and tools for Java developers that it was impossible for the MPS-based set of web languages, which the YouTrack team had created and used, to keep pace. The YouTrack team made a sensible decision to rely on proven web technologies, instead of maintaining their own ones, in order to make their development more efficient.

This only confirms that the sweetspot for MPS lies in domains outside of the programming mainstream, which cannot be well served with the mainstream technology stack.

Conclusions

I think everyone listening to your talks or screencast will be thinking the same question: how did you manage to get such a wonderful English accent?

Thank you :) It must have been through osmosis. I should perhaps thank all my British friends and colleagues for influencing me.

Anything else do you want to add?

Thank you for all the questions. I really appreciate your help in spreading the good word on language engineering.

Federico: it was a great pleasure for me! I have been interested in MPS for several years and having the possibility to ask questions to Vaclav has been amazing for me. I hope other MPS enthusiasts can also share my enthusiasm.

You may be also interested in: