MIT course "Security of computer systems". Lecture 11: “Ur / Web programming language”, part 1

Massachusetts Institute of Technology. Lecture course # 6.858. "Security of computer systems." Nikolai Zeldovich, James Mykens. year 2014

Computer Systems Security is a course on the development and implementation of secure computer systems. Lectures cover threat models, attacks that compromise security, and security methods based on the latest scientific work. Topics include operating system (OS) security, capabilities, information flow control, language security, network protocols, hardware protection and security in web applications.

Lecture 1: “Introduction: threat models” Part 1 / Part 2 / Part 3
Lecture 2: “Control of hacker attacks” Part 1 / Part 2 / Part 3
Lecture 3: “Buffer overflow: exploits and protection” Part 1 /Part 2 / Part 3
Lecture 4: “Privilege Separation” Part 1 / Part 2 / Part 3
Lecture 5: “Where Security System Errors Come From” Part 1 / Part 2
Lecture 6: “Capabilities” Part 1 / Part 2 / Part 3
Lecture 7: “Native Client Sandbox” Part 1 / Part 2 / Part 3
Lecture 8: “Network Security Model” Part 1 / Part 2 / Part 3
Lecture 9: “Web Application Security” Part 1 / Part 2/ Part 3
Lecture 10: “Symbolic execution” Part 1 / Part 2 / Part 3
Lecture 11: “Ur / Web programming language” Part 1 / Part 2 / Part 3

Nikolai Zeldovich: let's start, guys! So today we will talk about a completely different and principled approach to creating secure web applications. It's about a system called Ur / Web. Now our guest, Adam Chipala, who is the author of this system, a professor at MIT, will tell you about the system he created.



Adam Chipala: I want to get to the demonstration as soon as possible. But before that I will show some slides to explain the content of this system. You have probably already received some ideas about this from the summary of today's lecture.

So what is Ur / Web? It is always helpful to begin by explaining what the name of the topic means. Ur / Web is primarily a programming language for creating web applications. That's what the Web is in its title. This is a kind of full stack system that does everything you need to build web applications. Ur means the new universal programming language that is used to implement these web features.

The whole point of Ur / Web is that instead of using a general-purpose programming language, a library, and traditional frameworks for creating web applications, everything is already integrated into the custom Ur / Web programming language. It is a language that at run time includes compilation, not interpretation. And the compiler, in a sense, understands what a web application should do. It will indicate errors that you make, unlike the regular Java compiler, which does not understand where you have errors.

There are three basic principles that I tried to use when developing this language: programming efficiency, security and performance, especially on the server side, for scaling. In this context, the second is most relevant.

In most cases, users of your application will not notice small performance problems on the client side, but even a small problem on the server side can force you to buy many more servers than necessary.



At the moment there are already several Ur / Web users, although there are not as many of them as those who use other programming languages. But at least this is the only commercial web application that is an RSS reader and that supports such exotic features as displaying comments. There is also a URL, invented by a non-native English speaker, who now regrets such a name. It is called BazQux Reader, a combination of the tactical skills of the hacker community. This application already has several thousand paid users. And it seems that it is much more pleasant to deal with it than with what is done using CSS. This is proof that this can be done with Ur / Web.

Feel free to interrupt me with questions at any time, although I probably did not yet understand what might cause questions. Thus, the main success of sales of Ur / Web is that it has a very high-level programming model, which is very different from Django, which you know about from previous lectures. And he has a good security history.

Some features that you would like to have for security are already integrated into the system, so you don’t have to work hard to ensure your program’s security. I will tell you more about this in the near future.

In addition, this language provides high performance on the server side, even compared to the more popular tools for creating web applications, I think you have already heard about it. The caveat is that we will probably need to learn more ideas from functional programming languages, such as Haskell, before using Ur / Web.

I looked at the questions and answers for this lesson, and about a fifth of the students complained about the functional parts of programming that are difficult to perform. I'm sorry, but in the world of functional programming there are so many good ideas that it would be difficult not to start from this very moment, gradually adding more complex things to it. But I will try not to require strict knowledge of the material, which I will talk about today.

Thus, this programming model is indeed closely related to static typing. And this is not just static typing, as in the Java language, which has a relatively inexpressive clumsy type system, but static typing, similar to that used in Haskell or Apache Camel. This typing is one of the ways in which the compiler understands what you are doing and catches errors in its program.



It turns out that the main language Ur, on which Ur / Web is built, has a very expressive system of static typing. So much of what Ur / Web does actually provides libraries without special compiler support. For example, we taught the compiler how to check the type of SQL queries without creating SQL input rules in the compiler. They can be encoded as libraries and use the standard type of validation to ensure that your SQL queries follow the rules of SQL.

The most relevant in this context is ensuring a high level of security - most of the most common security vulnerabilities are not possible when programming on Ur / Web. You can forever say goodbye to malicious code injection attacks and cross-site scripting attacks. You can allow scary-looking flag names to do the most horrible things that can be done in a web application, even if you do invoke some kind of “black magic” using the functions of someone else's interface.

There are several other security-specific properties that I will discuss later. Performance is also very good. The compiler is primarily optimized for generating source code for domains and is much more productive than manual code written in C.

Therefore, he understands what the web application does and can optimize some things that a traditional general-type compiler would not “catch”. Therefore, the source code produced by this compiler, which runs on the server, successfully competes with the code in C. If you compare the cost of ensuring performance with the effort of programming in other languages, it is clear that Ur / Web makes life much easier for the programmer.

The next slide presents a fast benchmark for the performance of this framework for a third-party web infrastructure.

This is a screenshot of the results of the last test, in which different web programming tasks were performed by different frameworks.



You can see that Ur / Web took the 4th place out of the 60 frameworks that participated in the performance tests. After this screenshot was taken, some more improvements were made to the compiler. Therefore, I expect that in the next round, according to its results, it will move up slightly. In principle, this is a simple example of using SQL to generate HTML pages. You get about 100,000 requests per second from an Ur / Web server, and that’s more than enough for most applications. It is important that this slide suggests that you can use a high-level model that provides more security, slightly losing in performance to more common frameworks.

So let me start with pictures that reflect my impression that programmers today think of writing web applications using the most common frameworks. Then I will show another view that Ur / Web provides and according to which, a lot of things that can go wrong go on this framework without errors.

The main picture is the web server on which the entire process of your application depends. And there is a whole park of browsers that are going to interact with this server. The server will be in a certain state, which provides interaction with all of these browsers.

The usual pattern is that the browser starts interacting with the web server by sending it an HTTP request that includes the URLs embedded in it. After that, the web server sends an HTTP and HTML page back to the browser. However, there are some embedded URLs that can be used to determine what kind of request the web server should make in the future.



This web server can also communicate with a database that provides persistent storage common to all users of the application. It uses one popular SQL protocol for conversations between the server network and the database. This is what I will talk about when discussing the possibilities of Ur / Web.



Modern web applications are not just one single page. Whenever something has to change on the page, you make a new request to the server, after which the entire module page is replaced. There is an AJAX style, in which the browser, when viewed on one page, sometimes sends additional HTTP requests to the web server and receives responses that are processed by the user program. This typically uses data representations such as XML and JSON, as well as other simple formats for exchanging data between the client and the server.



Then, when the browser returns this response, some JavaScript code is executed there that implements arbitrary logic to control the user interface, which is shown to the user.



This javascript code can read the responses that the server gives to various AJAX calls. It can then change the page, which is basically displayed by changing the global DOM variable that is set for the page. Any part of the program can arbitrarily influence this global variable, which is the page. Often parts of the page are viewed with a string ID annotated with nodes of the tree describing the document.

And finally, another complication is that sometimes we want the web server to contact the browser without a request. Suppose a new e-mail message appears, and the web server wants to inform the browser about this new message.



And there are many ways to do this, for example, Comet is a model of a web application where a permanent HTTP connection allows a web server to send data to the browser without an additional request from the browser, or the WebSockets duplex communication protocol that communicates between the browser and the server in real time. In principle, they are the same things, but in a conceptually different direction.

So, I want to return all these protocols and languages ​​to the screen, pre-highlighting some parts in yellow. After reading the lecture notes, did anyone guess what is common between all these parts in terms of security?



Student: they are all lines in which you can put anything.

Professor:correctly, in the conventional approach to programming web applications, all these things are string. And the programming language does not understand how you use them, so it cannot help you avoid mistakes. So, for example, by representing these things as strings, you get code injection attacks. As far as I can tell, attacks with the introduction of code are mainly the result of the inclusion of some function as a primitive in your programming language or your framework that runs programs as text in a rather expressive language.

Ur / Web does not have a built-in interpreter for executing strings as programs. And it makes constructively impossible a lot of the most common errors of web applications. So all these selected objects will be either invisible or represented by special types, which makes it clear which code you are dealing with. In doing so, you do not have any automatic casting of strings to any of these special types.

Now the slide shows an alternative model that Ur / Web provides and which is compiled into a traditional model. This is how it works in all common browsers. But the programmer can think about it at a higher level and avoid potential mistakes that were possible in the previous picture.



Thus, we still have a web server that responds to requests. And we still have this park of browsers that are trying to use a web server. But the first important difference is that when the browser wants to start using a web application, it does not just send a string of HTTP requests with a URL.

It starts the first-class function on the server without invoking the client. And then the server responds not just with the text string of the HTTP protocol, but with a strongly typed tree of documents. Thus, instead of the HTML string, we have a tree, in a programming language - an object of the first class. And the program manipulates this tree, not the string.

Each of these trees contains links, which in themselves are just footnotes for other functions that can be called on the server. When the user clicks on these links, the browser selects the function and conceptually calls it on the server as the original function we called to get to this point.

We also have a database interface that is accessed by a web server that sends requests to the database. In the Ur / Web model, this is not just text, but strongly typed SQL syntax trees. And then the database will respond not with text, but with a list of eigenvalue entries in the Ur programming language with which we work.



Therefore, we don’t need to worry about improperly converting strings into native representations or about converting native representations into any other format that the database traditionally provides us.

This is a key element of how Ur / Web semantics make it easier for programmers to work with a large number of scripts that can actually occur while an application is running.
There is a standard transaction idea in the world of relational databases, where you can perform a series of operations without interrupting them with other parallel threads. And Ur / Web adapts this model and embeds it into the semantics of the language. Therefore, when one function is executed on the server on behalf of the client, all its access to the database occurs as an atomic unit without any interruption caused by all other simultaneous requests to the same server. You cannot avoid this behavior, even if you want, because these transactions are built into the programming language.

And they do make it easier to perform parallel queries and potentially help avoid security issues that arise when some kind of alternation of queries occurs alternating.

I want to get an answer to one of the questions presented in the abstract of this lecture and which intrigued me. So, Ur / Web detects when a transaction completes with an error due to such a parallel execution problem as a dead end, and automatically restarts the transaction. Someone, answering questions, wrote that it might make it easier to launch security attacks that depend on a transaction failure due to concurrency problems. I just wanted to ask the class if anyone could give an example of such an attack, how do you imagine it? If you have a system that automatically restarts transactions in deadlock situations, how can this cause security problems? I have no answer to this question, which is why I ask. This question may have such an unobvious answer that it is clearly worth discussing.



Student:Perhaps something like this could cause a DoS service failure? If he is going to restart the transaction that you are sending, and you know that it will not succeed, you can just continue to restart this process and try again ...

Professor: well, continue ...

Student: if you force the system to do what you know it will never succeed, you can repeat attempts again and again, and, eventually, cause a failure of the service.

Professor:correct, but in order to do this, you need at least two threads running simultaneously. Although this could potentially work and you will be able to launch a “service failure” attack. In this case, you can take advantage of the fact that request handlers are restarted again and again and deliberately cause a conflict, and use this as a way to increase the power of the DoS attack in addition to what you can get with the help of the traditional attack model of this type. Well, I can believe it.

Student: Is this the only way to cause a transaction to fail?

Professor: Yes, this is the only way to cause a crash and automatic restart.

Student:perhaps there is a third party that would conditionally fail. Then you could use this to monitor the behavior of other users.

Professor: you will also need a way to watch someone fail, but you will be able to do it only after a while. However, this can also be a problem. You can use a third-party channel to see what other threads are doing, because their actions might or might not create a conflict in your topic. This is possible in principle, but very intricate. I'm not sure that this is possible, because it is difficult to come up with a specific attack that will work predictably. But it can be a fun security test.



Student:transactions that you run for each incoming request, and you run for the code that runs on the server. But when you send this code to the database, does it turn into a database transaction?

Professor: yes, it is, all server-side execution is packaged in a single database transaction if the application uses the database.

Student: So, if you have a failed transaction, do you tell the database that nothing will be updated later? Because presumably, the database knows nothing about the failure of a transaction.

Professor:yes, so the compiler does static analysis and defines read-only transactions. This creates a read-only transaction, which on some database systems allows for additional optimization.

Student: how about the fact that some of the things you read do not affect what you are going to write, but other things you read can affect this?

Professor:you ask, can we use our knowledge of the semantics of the application to tell the database system that some things that look like concurrency violations are not really such that we do not need to restart the system? I think the answer will be brief - no, the current implementation of the language does not. But it would be interesting to consider in the future. I think that this will require a change in the database engine, and not just the programming language interface.

Student: you can usually divide it into two separate transactions, or maybe you can do so under certain circumstances?

Professor:Yes, it sounds difficult to implement, but it is potentially useful. True, I can not say how many applications can take advantage of this, but this is a neat idea. So a transaction is a great thing.

I was just telling you about an old school browser model requesting pages from a web server. We can also implement this process in AJAX style, which basically looks like client-side code.



This is a function call marked to run on the server. When it is completed, the result is returned in the client code and it turns out just a native value in the programming language. You do not need to worry about somehow turning it into a string and translating it back from the string.



After that we have to accept the result and use it to change the page that the user sees. Otherwise, it would not be a very useful request.

Thus, the model in Ur / Web is very different from the standard document object model that browsers provide directly. The main idea is functional reactive programming, I will not try to explain it in too much detail. Because I know that it requires nontrivial sudden functional programming, even if we cut off this reactive part. But the basic idea is a document that is described by a set of mutable cells, which are a kind of data and on which the page depends.

And the page itself is something else, which is described as a function that takes as input the values ​​of these cells, and then the calculating page. Further, the language system monitors the changes in these changing cells as they work, and when they change, it automatically calculates the consequences for the displayed page and effectively updates only the part of the page that has changed based on these cells.

Many different threads can be running on each client at the same time. These streams are born in the Ur / Web code and run the Ur / Web code themselves. But the compiler needs to translate them into JavaScript so that the browser can run them. So this is one of the types of services provided by the compiler. This is an important moment in the streams.

Another key point is that streaming processing on the client side follows what is called a collaborative multi-threaded model. The flow does not have to worry about what can be forced out by another flow to an arbitrary point. There are well-defined operations that signal that everything is in order to switch here to another stream. One of such operations makes a remote function call on the server, or, for example, asks to freeze for a certain number of milliseconds.



But ordinary code cannot be stopped arbitrarily. This means that the programmer does not need to think about a lot of alternations, and it is easier to convince yourself that, they say, a specific piece of code allows you to avoid some security problems and errors. Because you can more easily list all the possible ways that the two threads interact with each other. This is a kind of natural model to use, given the way JavaScript is usually implemented.

In JavaScript and browsers, there is still no crowding out, so this is simply a representation of stream abstraction over a callback-based model that JavaScript directly shows to the programmer.

And the last part, which is one of the built-in abstractions used by Ur / Web applications, is the channels for transferring messages between different machines. Thus, each channel has a type that expresses what data can be transmitted through it. You do not need to convert the data into strings and back or turn it into a textual JSON data exchange format. These channels can live in a database.



Imagine that this picture shows us where the created channel is. He has a recording side and a reading side that can go to different places. The end of the record is in the database. And the read end somehow makes the way to the client and is in the variable of the flow environment.

So, imagine that the thread previously made a remote call to the server that created the channel, returned it to the client, and placed it in the database in one transaction. So later the server decides: “OK, I’ll request this channel from the database and add value to it,” and it seems to pop up from the other end of the channel to the client. And everything in this system is “tied” to a similar process.



I think this was the last step in the demonstration of my pictures. Do you have questions about this model before I switch to code demonstration?

Student: why do I need to pass the message if the request is automatically processed by the server?

Professor: The RPC interface is about to initiate a call from the browser, and the server handles it. A message occurs when the server itself initiates communication.

A canonical example is a new e-mail message that a customer is expecting. It cannot determine when a new message is available, so the server automatically sends it.

Student: all messages are multiplexed through one connection or through different connections?

Professor: they are all multiplexed over an HTTP connection. I know that there are these newfangled things called web sockets and maybe some other protocols, but here everything works on old-fashioned HTTP with one connection for all messages via different channels.
So, see what happens next. Let me switch to the demo. So welcome to the Ur / Web program. As you can see, so far nothing terrible.



Unusual here may be that it is really a whole program. There is no additional routing logic explaining how to match the URL with the code for querying this URL. We have only regular functions of a standard programming language. And the compiler provides all the functions in the main module, which is called via the URL.

The URL is simply formed from the name of the function. And if there is any nested modular structure, it is also replicated in the URL. Then we have a function that returns HTML syntax. The compiler uses a special extension for parsing HTML syntax. It also performs some basic checks to make sure that the various XML elements that are displayed inside others are actually authorized for such a display.

I did something in advance, and it does nothing surprising in the browser. This is what the HTML page looks like among other properties, it automatically adds the page title and declares the character encoding for this document.



I was a little shocked looking at some books intended for reading in this course. I’m surprised how much time is spent in talking about character encodings and what happens if you don’t use UTF-8 encoding. I hope I understood that correctly.

I hope they make you use UTF-8 so that no terrible things happen. But if someone sees a way to reproduce any of the attacks described in the book Tangled Web (Tangled Internet) on the Ur / Web server, I would be interested to hear how he is going to do this.

By the way, at any time of this demonstration, please suggest experiments that come to your mind so that we can try out on my system what kind of errors it is capable of catching.

I think this is the most productive way to demonstrate the capabilities of Ur / Web.

27:45 min.

The course MIT "Security of computer systems." Lecture 11: “Ur / Web programming language”, part 2


Full version of the course is available here .

Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending to friends, 30% discount for Habr's users on a unique analogue of the entry-level servers that we invented for you: The whole truth about VPS (KVM) E5-2650 v4 (6 Cores) 10GB DDR4 240GB SSD 1Gbps from $ 20 or how to share the server? (Options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).

VPS (KVM) E5-2650 v4 (6 Cores) 10GB DDR4 240GB SSD 1Gbps until December for free if you pay for a period of six months, you can order here .

Dell R730xd 2 times cheaper? Only here2 x Intel Dodeca-Core Xeon E5-2650v4 128GB DDR4 6x480GB SSD 1Gbps 100 TV from $ 249 in the Netherlands and the USA! Read about How to build an infrastructure building. class c using servers Dell R730xd E5-2650 v4 worth 9000 euros for a penny?