Bromium DSL: a DSL to Test UI Actions

Bromium DSL: a DSL to test UI actions

 

I recently heard about Bromium, a Domain Specific Language (DSL) to describe user actions on a UI. Given I love everything regarding DSLs I contacted his author, Hristo Vrigazov, and chatted with him about:

  • the DSL he has built,
  • building editors using Xtext,
  • working on an open-source project
  • his visions on language engineering

Enjoy!

P.S. Hristo had an issue with his webcam so you will see just me in this interview

 

Introduction

Federico: Thank you very much, Hristo, for joining us and to agree to this interview. Maybe the first thing you can teach to who is listening is how you pronounce your name better than I did. Can you say your name?

Hristo: My name is Hristo, I think you managed to pronounce it correctly so thanks.

Federico: Can you tell us a little bit about yourself, like where are you from? What you’re doing?

Hristo: Okay. I’m computer science student in my last year. I’m located in Sofia, Bulgaria. For the past three years, I have been working on QA automation. I have been testing a product which is related to eCommerce artificial intelligence. It included both, lots of infrastructure as well, but also UI testing.

Federico: Okay. Testing, maybe on different browsers? Things like that?

Hristo: Yeah. Also, using Selenium to automatically perform tests, to save manual effort. I’m actually currently moving to another industry. From October, I’m going for work in autonomous vehicles. It’s not a big jump as it looks, because I’m in my university years, so I constantly learn a lot of stuff and computer vision is sort of my passion.

Bromium: A DSL to Describe UI Actions

A quick demo of Bromium in action

 

Federico: That’s a very interesting topic, yes. Good, good. It seems you have interesting things ahead, good. Today, I wanted to talk with you in particular about one of the interesting things that you have been working on, because we got in touch through my newsletter. We discuss about Bromium, that is your DSLs to describe user actions. I think it would be nice for our readers to understand what kind of problems you were trying to solve using Bromium and also, why you chose to write a DSL for that. Because, I think people have some difficulties to understand when it’s useful to use a DSL, in the sense that maybe you could have just wrote an application with some configuration mechanism or whatever. Can you just introduce the project?

Hristo: The situation that I had at work was the following: we had a lot of manual QAs, which had a very good understanding on the project. Really good understanding of how the different components of the project work and so on, but we didn’t have as much automated UI tests. The reason for that was the application used a lot of Ajax calls and basically, dynamic stuff that didn’t allow us to write those tests.. The automation wasn’t as simple as for example: find element by ID and click on something, because, for example, it wasn’t waiting for an element in the DOM to arrive. Things like that. For example: There are automated ways to listen for elements when they arrive, but it’s this logic that they’re not interested in. It’s not business logic. I tried to separate the business type, basically the contract of what we can do. A contract of what we can do with the UI, from the actual browser automation stuff.

Federico: Okay. It seems like a very high level, it seems that you’ve used the DSLs to separate, in a way, the contract of what you want to do from the implementation. That, in a way, maybe it is part of the interpreter.

Hristo: Yeah.

Federico: Instead of business logic, there is this term that Markus Voelter is starting to use, that is Fachlichkeit. I guess, it’s German so it’s very difficult for me. He’s trying to introduce it in the field, so we have to start using it. Okay, good. What about the history of the project? How old is it, how mature is it? Have you already used it for active testing or its more like a prototype, an experiment?

Hristo: I decided to start a project outside of work, because it didn’t look like this is something that we have the time to invest at work.

Federico: Okay.

Hristo: I started about March, 2017. How mature the project is? After the first two months, there was the core logic which was sort of difficult to be written. After that, we have extended it with more actions. We were immediately able to use it when working on some tickets, so we have used it for writing some tests. Also, recently we used it for a big performance test of the product. Basically, we were simulating a lot of different users interacting with the product.

Federico: Okay. I think here, it’s interesting the fact that you, if I understand correctly, you’ve built that initial version on your own? Not during work time, so you didn’t have to convince any manager to give you the approval.

Hristo: Yeah.

Federico: You avoided the big problem of explaining what are the advantages of DSLs and convince people. That it’s something that I’m still struggling with. I mean to understand what is the best way to convince people to go for DSLs. Because, in my case, clients typically contact me when they have already decide they want the DSL, but I probably wouldn’t be able to convince many companies. The explanation part for someone that is not familiar with DSL is very complicated, but in this case you did the investment so you avoid the issue. You said for the first version, a couple of months to build the logic?

Hristo: Yeah, it sort of … Because, the setup is a bit complicated, I can later explain but it’s sort of a logic that has to be used in every run and then, you can extend it with actions.

How the Bromium DSL looks like

Federico: Okay, but in this phase you were mostly working on this logic if I understand correctly, the logic to execute the action more than the DSLs itself. It wasn’t two months spent working on the grammar of the DSL?

Hristo: Okay. I didn’t have the DSL from the beginning, this is actually something that I probably should have mentioned. Initially, it was just JSON files or YAML files. I was thinking I want to get this out as a configuration, that’s how I thought I did at the time, but the problem is that it was really ugly. Also, it didn’t have things like auto complete. It was really a pain to edit JSON files. At the same time, I was taking a course in university which was related to programming languages and this is when I just realized that using these JSON or XML, or whatever, is sort of like editing directly the AST. That’s how it felt, because that’s why it was so inexpressive. Actually, I still have examples and I can show, for example: the JSON file has like 90 lines when it’s formatted so that I can read it, compared to like 10 lines when writtin in the DSL.

Federico: Okay, then maybe later we should take this example and put it in the body of the article to give an idea?

Hristo: Yeah.

Federico: JSON versus DSL.

Hristo: Yeah, that would be cool

Here we can see an example of a JSON file used to configure Bromium before the DSL was introduced.
Below you can see a little snippet of that JSON file:

Using the DSL is much more clear and concise. You can find an entire example here. Below you can see a snippet :

Much better, right?

Federico: Yeah. Okay, so you started. You get something done, you start using configuration and then you realize that if people are using this all the time it would be better to find something else. Maybe, they are not technical people. Instead of editing a JSON file, try it and see it explode, it would be better to have a DSL with an editor that help them while writing it. I guess, can also point out some errors.if

Hristo: Absolutely.

Federico: That’s already a good distinction between configuration versus DSL. Why you decided to build this tool? Didn’t you find any available solution that you could just reuse? Aren’t there any system to record the interactions and just replay them?

Hristo: There is a tool called Selenium IDE which is a Firefox extension. It records interactions, but for example: If the DOM element has auto generated ID, it would record it as click on ID and it just doesn’t know about the app specific actions. It doesn’t allow it to specify them in any way. While, with Bromium, you say for example: A click on button with ID login, for example, translates to an action in the app which means click login button. Now, when it clicks, it records a domain specific action. The problem with Selenium IDE is that it records on a browser level and if, for example, the ID was auto generated or depends on date and all this stuff, it won’t happen. It’s not reproducible, that’s what I’m trying to say.

Federico: Okay, I understand. I guess in this way you decouple the intention from the actual action, like a click to submit button from the the technical details. This is the ID of the submit button, so that maybe if at some point in the application the ID of the submit button change, you have to update your definition of click on the submit button.

Hristo: Yeah, absolutely.

Federico: You don’t have to replace that all over the place, okay. That seems like a good advantage of this tool. The people that are using the DSLs, they can just write something that sounds meaningful to them, like fill the name with “John”, then, click the submit button.

Hristo: Absolutely, yeah. That’s exactly one of the main advantages.

Federico: Can you tell us a bit more about the kind of users that are working with this tool and with this language? Are they testers?

Hristo: Yeah, they are mainly testers. Basically, to work with the language you just have to know the product well and the product that you’re trying to test, so that you can design the contract. Also, you have to understand CSS selectors, which is pretty basic for a tester.

Federico: Yeah, okay. It’s maybe not exactly someone that doesn’t know anything about technology. The typical user has some understanding but is not someone that necessarily to be a good developer to use the DSL.

Hristo: Yeah.

Federico: Another thing I will be interested in discussing is the design of DSLs. Was it difficult for you to come up with the syntax? Did it take a lot of trial and error? Anything you want to share about designing the DSL?

Hristo: Basically, I had already had the JSON configuration, so I had an idea of how it’s going to kind of look. It’s not exactly the same obviously, but I had an idea of how I wanted it to look. One thing that I decided was due to the target users of the language, which are as we said many QAs, mainly I wanted it to be as close as possible to English, just normal English. Which, at the time, seemed like a really good decision and I think it has its advantages, but it also has some disadvantages because one of them is that you get so many keywords. That, you can have some collisions.

Federico: Yeah, you cannot use that in a normal sentence.

Hristo: Yeah, but it’s not… It’s not that bad and I think it’s probably the best decision, because otherwise people that were going to use it are not familiar with C based languages, for example. I think that it was better to do it like this.

Federico: Okay. If I understand correctly, in your case was a big advantage. The fact that you have first defined the logic to configuration file, so you knew already very well which concepts you want to express in the language. That’s already a very good starting point and then, you have just to define the syntax. But, the abstraction level was already the same of the configuration file, so you knew what you needed to express. You just have to define the syntax and you went for the natural language syntax, sort of?

Hristo: Yeah.

Building an Editor for the DSL

Federico: If I understand correctly, you have built your editor using Xtext, right?

Hristo: Yes.

Federico: Was it a difficult decision, did you evaluate other options or you found was an easy choice to go for Xtext?

Hristo: I had a bit of research for tools. First of all, I must say that I’m an IntelliJ fan. I prefer it over Eclipse, so I research and basically the two options were MPS, which, was this in IntelliJ thing and Xtext, which was kind of tied to Eclipse. There were some advantages and disadvantages, but in the end I decided to go with Xtext. The reason was that MPS is not exactly the same thing, it doesn’t store the files. Xtext takes this approach where you write normal text files and from then, it creates the AST and you write an interpreter or whatever. I’m not super familiar with MPS, because I haven’t used it, but MPS looked to me like it stores the files in this format unreadable for humans. It looks like it stores them in AST like format and then uses some fancy editor to show them nicely to the developer. This has a number of disadvantages, at least in my view. One was that I can’t easily send it and read it without special editors. Also, the version control looked kind of different, although I read recently that they have some special diff tools, I think.

Federico: I understand what you’re saying, because in the past I worked mostly with Xtext. Now, I’m working mostly with MPS but I see your point. Probably, for people with technical background, like testers, there are advantages of having a tool like Xtext, that just save text files. They save the file in the same format that they user look at the file, while MPS is a projectional editor so it saves these very large XML files. Then, read it and draw something on the screen for the user to see, but what the user sees and what is saved on the file is very different. As you said, you can only edit these files using the MPS editor. If you want to send the files to someone, to do a quick change, you cannot do that or sending a correction in an email. Doing version control, it’s also a bit complicated when you are inside MPS. You can do merge using a sort of visual tool, so that works, but if you are used to do everything from the command line, no.

Hristo: Yeah, that’s exactly what I was going to say.

Federico: Yeah, I also agree on the fact that I used to like very much Eclipse, but in later years I have becoming more a fan of JetBrains. I am also wearing the Kotlin T-shirt right now. They are doing amazing stuff. When I started looking at Xtext, I found very difficult to get started. How was your impression, do they now have better documentation? Was difficult to write the first language?

Hristo: Actually, one of the things that I really liked about Xtext was that it looked like it’s not completely locking me into Eclipse because at least on their site they were saying at the time, that they canalso generate IntelliJ plugins.

Federico: Yeah.

Hristo: I was thinking I can live with using Eclipse, the point is that at the end, I’ll have editor for IntelliJ. But, it turned out that it’s not maintained really, it only worked with old IntelliJ versions.

Federico: Okay.

Hristo: Yeah. Other than that, I think I started from a 15 minute tutorial to Xtext or something like this. I got some initial idea and then, there was another article which was describing the Xtext grammars, in more detail which was really fancy. I just played a bit with it. Also, there as a book that I haven’t fully read, but I just skimmed through it. That’s how I got started with it. Also, the DSL is relatively simple language so it doesn’t have even things like variables and types, and so on.

Federico: Okay. Now for Xtext there is a good book from Lorenzo Bettini. I forgot the name of the book (note: it is “Implementing Domain-Specific Languages with Xtext and Xtend”), but I also wrote a review of it. It’s very nice. The problem that had also when I started the Xtext was that at the beginning, if I wrote a simple language, it was mostly defining the syntax of the language. That was okay, but when I try to do slight more complicated things and I need to understand the Eclipse Modeling Framework (EMF), then it took me quite some times to understand it.

Hristo: It’s really locked. There was something like this, after you’re talking about the EMF model that is built when the text is parsed, right?

Federico: Yeah.

Hristo: There weren’t many resources for that, I think.

Federico: Yeah.

Hristo: At least, there could be but I either didn’t look for them in the right place or couldn’t found them for some reason.

Federico: Yeah, but if they exist, but someone cannot find them, then it’s not really important that they exist. Which feature did you build in the editor, can you describe a little bit your editor?

Hristo: Apart from the built-in auto complete, I had some custom autocomplete and suggestions which were relatively simple. I think it’s visible actually at the YouTube video that I uploaded. It is a two minute introduction to Bromium (note: we included the video above in the article).

Federico: Perfect, we will put that also in the article for users to get their first impression of the tool.

Hristo: Yeah. That’s the custom validation, but … Yeah. It’s validation and autocomplete, but I think that there were in only for the Eclipse plugin. It looked really extensible and very convenient, but I think it was mostly for the Eclipse plugin. While, for example: those custom autocompletions that I entered didn’t work, for example, for the web plugin. It was working in a completely different way, I think.

Federico: The web plugin… because are you generating both the editor to be used in Eclipse and the editor to be used on the web?

Hristo: When I was creating the project, I think it prompted me at the beginning. I had different projects and I think that there was a separate project for the Eclipse plugin, and a separate one for the web editor.

Federico: I just didn’t understand if the web editor was something that you built without Xtext?

Hristo: No, I used the generated project and iterated on them.

Working on an Open-source Project

Federico: Okay. I wanted to ask some question about the opensource project, because I am very much interested in opensource and understanding how to make opensource projects work well. I was curious why you decided to release this project as opensource?

Hristo: The main reason was because I really liked the idea of having this contract and since I was working on it in my free time and I wasn’t sure that I’m going to have the time necessary to develop it in a really nice form, so I decided it’s better to put it as opensource, and just hope that for example: If someone really likes it, it can grow.

Federico: I know that at the beginning of a project, it’s very difficult to find users. Especially, if your product is something that is not similar to something else that already exists, because in this way people don’t look for it. A project that in contribute to, JavaParser, I think is very successful because of its name. Because, every time someone look for “Java parser”, he finds JavaParser. Very simple.

Hristo: Very descriptive.

Federico: Are you starting to see some users opening tickets and asking you for change or it’s a bit early for that?

Hristo: People who know me and I have talked to them personally about the project have expressed interest. The project it’s not that active currently and I’m currently looking for ways to just… It’s not something that is super useful for everyone, but I think it can really accelerate testing. I really hope that in the future, there will be more interest.

Federico: It definitely seems like something very useful. In the past, I worked at companies where we were building web applications. I worked at Tripadvisor and Groupon, and it was a major task to test that, for example, everything was working on different browsers. Even ancient versions of some browser, so something like this seems very interesting to add. Let’s hope that some of our readers help us spreading the voice about the project. Sometimes, when we talk about opensource projects, I ask if you have a strategy to monetize that, but maybe in this case it’s a bit early or have you already start to think for a way to make money out of your project one day in the future?

Hristo: I haven’t really thought about making money on this particular project, because I’m kind of more interested in other things at the moment. Which, again, are probably going to be related to language engineering, but I haven’t really thought about monetizing it.

Conclusions

Federico: Good. You nominated language engineering, so I was curious if there is anything in the field that you find particularly interesting and you would suggest people to take a look at?

Hristo: There are a lot of interesting stuff, but the main thing that I’m interested in is language engineering combined with some code generation for different targets. Just have this unified … like an interface to other platforms and so on, for example: when I was at the interviews for the autonomous vehicles job, when I mentioned that I had some experience with just building a simple language. They actually said that it’s actually really good and actually, they were using Xtext.

Federico: Nice.

Hristo: It’s actually a company which builds those products, but for different manufacturers. They have a lot of different targets to which they want to generate code and I guess they’re using something like this to unify all of those.

Federico: Okay.

Hristo: That’s the main thing that I’m interested in for language engineering.

Federico: You say writing languages and combine them to code generation, if I understood correctly?

Hristo: Yeah, that’s mainly. Also, cost of this starting validation and doing some reasoning about it. With JavaParser or a simple parser.

Federico: Yeah. With JavaParser, I have the impression that parsing Java has a reasonable complexity, but doing the reasoning and this includes doing all the symbol solving. For example: Understanding that if you are referring to A: In a certain context, A is a field, is a parameter. It’s a local variable, that it’s much more complex. At least, in a language like Java where there are generics and lambdas that interact in weird ways. It’s where most of our bugs are. e met each other through my newsletter, so I think you are reading our blog and I would like to ask if you think there are something we can do to improve the blog or some topics that we should focus on more?

Hristo: Some of my favorite blog posts are the ones about parsing any language. Actually, one of my side project I was recently creating a home security system on a Raspberry. I sort of had to sync the communication between a Python thing on the Raspberry and the server I brought it in Java. It wasn’t using HTTP, it was using some custom format. It’s a long story, but basically I wanted to make sure that those interactivity parts are generated from the same contract or whatever. I really loved the parsing a language in Java blog post. That’s really good, for Java I still haven’t looked but I’m sure that JavaParser has more high level things, but it’s good enough for other languages which I really liked.

Federico: Thank you. My final question, do you have anything else to add? Maybe something I forget to ask and you wanted to talk about?

Hristo: Actually, when I created the plugin for the Bromium tool, one thing that was kind of different was how to actually get it to people. Because, they were all using IntelliJ, the web editor was okay but it didn’t have some of the more advanced and convenience features that I created. I actually tried to figure out how to add them as well, they use the Ace editor which is used in Cloud9 IDE, and so on. But, I didn’t manage to get as deep into it as I wanted, so in the end I had to give to my colleagues the Eclipse plugin, but they weren’t using Eclipse. It had all these weird problems. If it’s not configured correctly, I decided to package it as a docker image. It’s a docker image, which … Since it’s a docker image, I know it works every time because it’s just virtualization. It was executing the Eclipse on your screen, not inside the container obviously, because you won’t be able to see it. But, there were some issues. Since, at work we are all using Ubuntu, this probably won’t work for Windows users. For Macs, it will work but anything that isn’t X11 based for screens wasn’t working. It was good enough for us, so one thing that I’m currently not sure was how exactly to choose it. You have something that is just you run it, you see it and if you don’t like it, you can delete it and so on.

Federico: Yeah. That’s true that it’s also one thing to consider in general when we build DSLs. The way that we distribute the plugins, or the editor. They are typically not so simple pieces of software, so they require parts and configuration. They could have difficulties running on some machines, so using a docker image seems a good way to avoid all this issue and fix that.

Hristo: Yeah, but it has the downside that Windows users won’t be able to use it. Actually, I think there were some X11 clients for Windows, but at the end it’s going to be more complicated to configure it than just download Eclipse.

Federico: Probably, that’s the case. Okay, that was my last question. I want just to say thank you again very much, for agreeing to this interview. Also, for releasing as opensource your project, I think it sounds very interesting both for people that could just use it, but also for people that are interested in DSLs. I think it’s interesting to hear your story and get inspired to build their own DSL. Hopefully, we will inspire some young language engineer to get started.

Hristo: Yeah, the language itself is super simple. It’s probably like 90 lines, the grammar. Thank you for having me, it’s been really cool to talk to you in person. Yeah, thank you. Thanks a lot for having me.

Federico: Thank you, bye.

Final Words

That was fun! I hope you enjoyed it as much as I did. If you think I should chat with someone feel free to contact me and share your suggestions

Download the guide with 68 resources on Creating Programming Languages

68resources

Receive the guide to your inbox to read it on all your devices when you have time

Powered by ConvertKit

Do You Need a Domain Specific Language (DSL)?

We can design and implement languages tailored to support your processes. We build also all the necessary infrastructure: editors, code generators, compilers, simulators. Our goal is to deliver complete solutions.

We can use different technologies like Jetbrains MPS, Xtext, and ANTLR for custom solutions.