Overview of the new features of Mathematica 11 and the Wolfram Language

Translation fasting Stephen Wolfram (Stephen by Wolfram) " the Today the Launch for We the Version 11! ".
Many thanks to Polina Sologub for help in translating and preparing the publication.

---

Content

- The first thing you note ...
- 3D printing
- Machine learning and neural networks
- Audio
- Built - in data about anything: from the skeletal structure and food to information about our Universe
- Calculations with real objects
- Advanced features of geographic calculations and visualizations
- Do not forget about the complex tasks of mathematical analysis and theoretical physics ...
- Education
- Combining all functions into one whole
- Visualization
- From lines to text
- Modern approach to system programming
- Work on the Internet
- Cloud data
- Connect to any external services: Facebook, Twitter, Instagram, ArXiv, Reddit and many others ...
- WolframScript
- New in the core language Wolfram Language
- And much more ...

---

I am pleased to announce the release of the new version of Mathematica and the 11th version of the Wolfram Language , available for both desktop computers and in the cloud. Over the past two years, hundreds of people have worked hard to create it, and several thousand hours I myself. I am so excited; it is an important step forward, essential for many of the largest technology areas. More than 28 years have passed since the first version was released - and almost 30 years since I began to develop it. And all this time I continued to realize the daring dream - to build an ever larger and larger stack of technologies



. Most of the software after several years and several versions, with the exception of minor improvements, practically does not change. Mathematica and the Wolfram Language have a completely different story: over the course of three decades, we have moved forward with each new version, gradually gaining a huge number of new areas .

This is an amazing intellectual journey for me and for all of us. From the very beginning, we had a serious set of fundamental principles and a strong basic structure , so that for three decades we could simply continue to build on this foundation more and more, creating everything that has now become a huge systempreserving its unity, elegance, and, frankly, modernity. In the early years, we focused on abstract areas like mathematics . But over time, we embraced more and more types of computing and knowledge .

Each new version is a lot of new ideas and many thousands of hours of hard work. Moreover, it is an ever-growing leverage created using our technology. This is because one of our key principlesis automation, and in each new version we use all the previous achievements from this area - so, in essence, we have more and more building blocks thanks to which we can move on faster and faster. Great efforts have been put into creating and maintaining a single structure for the whole system for many years, so that as a result all these building blocks from different areas are ideally suited to each other.

Using traditional software development approaches, it would take us many years to create what we added to the 11th version. And the fact that we can present a new version now is a direct reflection of the effectiveness of our technology, our principlesand our methodology. I am very pleased to see that we have come so far not only in terms of the contents of the system, but also in how effectively we can develop it. Not to mention the fact that all these areas that we have been pursuing for many years as part of the logical development of our system have now turned out to be exactly what is needed for many of the most actively developing technological areas.

For many years, we have called our core Mathematica system. However, after we added new directions, and also went far beyond the bounds of things connected in any way with “mathematics,” we decided to introduce the concept of Wolfram Language to convey the essence of everything that we do. And now Wolfram Language defines the work of not only Mathematica, but also the Wolfram Development Platform and the Wolfram Programming Lab , as well as other products and platforms. And thanks to the unity of our software development, we can today release the 11th version of all our Wolfram Language-based systems. OK, so what's so grandiose and new in the 11th version? Well, we have prepared not just one big innovation, but a lot. So you can appreciate the scale: we have added 555 completely new functions , representing a huge number of new functionalities (compared to the first version, which had 551 functions in total



) In fact, even in-depth versions of many existing functions are not included in this number of functions.

We manage the development of the company in such a way that we always have a portfolio of relevant projects - from small enough to those that may take five or more years to complete. The 11th version includes the results of several five-year projects. We are interested in conveying the results of our research to users as quickly as possible, so that since the release of the 10th version we had several intermediate releases; as a result, the 11th version is actually a combination of many completely new developments that we have already announced in versions 10.1 , 10.2 , 10.3, and 10.4 (many features that were flagged as “experimental” in 10.x releases are now fully available in version 11.0).

The first thing you note ...

When you first launch the 11th version on the desktop, the first thing you notice is the new appearance of documents with sharper fonts and a strict design. When entering the code, many new autocompletion options appear (it is becoming increasingly difficult to type something incorrectly), and when entering text, a new real-time spellchecker works , which we will constantly update to be sure that the last words are included in it .

If one of a dozen other languages ​​other than English is installed on your computer, you will see that each function is automatically marked with a “ signature ” in your chosen language: When you run the code, you will notice that the messages



also look different, and, which is very convenient, they allow you to immediately see what functions were called during message creation.

3D printing

The 11th version has many new areas. Let's move on to one of them: 3D - printing . I first printed in 3D back in 2002. For many years, we have exported to STL . For the 11th version, we created a complete cycle that starts from creating 3D geometry and ends at the moment when the finished product lies on your 3D printer. I used to want to take a drawing in 3D and just print it - also in 3D. And sometimes I was lucky and it was easy to implement. But in most cases, this is inconvenient and difficult, since the graphics that are displayed on the screen do not necessarily correspond to the geometry that can be printed on a 3D printer. So it’s convenient to configure or fix



geometry so that it really works on a 3D printer is a difficult task (and yes - if you do it wrong, plastic can splash out of the printer).

In the 11th version, it became possible to take any 3D-graphic and simply print it in 3D . Or you can get the structure of the molecule or the elevation around the mountain , and also just print them in 3D. Over the years I have printed a lot in 3D, and each time it was a kind of little adventure. But now, thanks to the 11th version, it’s easy to print almost anything in 3D. And now, when I think about it, I think I need a 3D printout of how Wolfram Language grew in different areas ...

Machine Learning and Neural Networks

In a sense, Mathematica and Wolfram Language have always dealt with artificial intelligence (AI). Over the years, we have been pioneers in solving a series of problems related to AI: from a mathematical solution to the automation of aesthetics to understanding a natural language. But back in version 10, we took a big step forward in relation to machine learning - we are talking about developing the most automated main functions ( Classify and Predict ).

I must say that I did not understand whether these functions would work well in practice. It was wonderful to watch how well they work., - and it’s very nice to see that many of our users can implement machine learning in their work exclusively with the help of automation developed by us and without consulting with machine learning experts.

In the 11th version, we pretty much advanced in the field of machine learning . Now we have new ways to carry out not only classification and forecasting, but also operations of feature extraction , size reduction , clustering, and so on. And we also “train” our system a lot to provide users with ready-made machine learning functions. Machine learning is a new and exciting development area. This is essentially a curation process. It is as if instead of, say, collecting data on various videos , you are collecting as many images of various animal species as possible.

The 11th version has built-in functions such as ImageIdentify , which identifies more than 10,000 different kinds of objects . Due to the general structure, it is quite simple to take the already highlighted characteristics and use them to train new image classifiers more efficiently than before.



We have done a lot to automate the most common machine learning tasks of today. But only in recent years it became clear that with the help of modern methods of machine learning (and, in particular, using neural networks) many problems can be solved. This is a truly amazing episode from the history of science : the area of ​​neural networks that I have been working on for almost 40 years has become one of the hottest areas from the seemingly almost hopeless one in which new discoveries are made almost every week.

Well, well - let's say you also want to do this. Yes, you can collect something using a number of low-level libraries. But when creating the 11th version, we set ourselves the goal of creating an optimized symbolic way to configure and train neural networks in which as much as possible of what they can do in principle should be automated. The 11th version there is now a function like NetGraph and NetChain along with all sorts of " special features of neural networks " type DotPlusLayer or ConvolutionLayer . And with the help of these functions, you can take the latest networks and quickly configure them in the Wolfram Language (recurrent networks in the 11th version are not completely completed yet, but they will appear in the near future).



Of course, all this works well thanks to integration with Wolfram Language. A neural network is just a Graph object, like any other. And input data such as images or texts can be immediately and automatically processed using standard Wolfram Language features in forms suitable for neural network computing.

From their name ("neural networks") it seems that they are connected to the brain. However, in reality these are completely general computational structures: they correspond to complex combinations of simple functions. They are not related to the simplest programs that I have studied for so long , although they have special characteristics that are configured so that they can easily learn by examples.

We always had functions of statistical data fitting and interpolation . However, neural networks have a much richer space of possible computational structures for data fitting or training. It is remarkable that only in the last couple of years, so many different areas have undergone revolutionary changes - and soon there will be even more.

I hope that we can accelerate this process with the 11th version. We managed to make "neural network programming" just another programming paradigm, along with the others integrated into Wolfram Language. This is very effective and allows you to interact with huge learning sequences. But in the end, I think the strongest thing is that it immediately fits into everything Wolfram Language does. Even in the 11th version, we already used it when working with our internal algorithms in such areas as image processing , signals and text . The history of "neural network programming" is still very young, and I am glad that Wolfram Language will play a central role in it.

Audio

Ok, let's turn to another new area of ​​the 11th version: audio . Our goal is to be able to process any data directly in Wolfram Language. We already have such objects as graphic primitives (from which you can build arbitrarily complex vector 2D and 3D graphics) , raster images (also 2D and 3D) , geometric calculated objects , graphs and networks , formulas, and many other functions presented in series in the language in the form of symbolic structures. And, starting with the 11th version, another data type is now available - audio .



Audio is difficult to work because of its size. However, in the 11th version, you can easily process, say, an hour of audio directly in Wolfram Language. Of course, behind this are caching, streaming, and so on. But it’s all automated - and in the language it’s just an object defined by the Audio function . This Audio object can be immediately subjected to extremely complex processing and analysis, which are available only in Wolfram Language.

Built-in data on everything from skeletal structure and food to information about our universe

Wolfram Language is a knowledge-based language. And this means that a lot of knowledge is built into it - both about calculations, and about everything in the world. Today, Wolfram Language covers thousands of areas of real-world knowledge - from countries and films to companies and planets. All the new data goes to Wolfram's main cloud-based knowledge base , and we carefully oversee the data about new things that are appearing in the world (who knew, for example, that recently new administrative units have appeared in Austria?). Many of these data are available at Wolfram | Alpha . But only in Wolfram Language data comes to life for complete computing- and then all the efforts that we put into ensuring the coherence and consistency of the whole system become apparent.

We are constantly working to expand the scope of knowledge covered by Wolfram Language. Those several areas that we have been working on over the years in the 11th version are finally ready to work. It was especially difficult with data on anatomy . However, in the 11th version you will see detailed 3D modelsall significant structures of the human body. You can see how the complex leg bones fit together. And you can make the calculations. Or print them in 3D. You can study the network of arteries around the heart. I must say that when I explored this, I was more than ever struck by the level of morphological complexity inherent in the human body. However, now we can easily make calculations in this area. New, and possibly unexpected, features have appeared to serve them, such as AnatomyPlot3D . Of course, they still have to work with them: for example, our anatomical data are only the data of the “ average adult man ”, and the joints cannot move, etc.



A completely different area of ​​data is also being finalized at the present time in Wolfram Language - this is food. There are many difficulties in this area. First, ontological problems. What is an apple? There is a general concept - “apple”, and also there are many specific types of apples. Also, the question arises of determining the "amount of food." A cup of strawberries . Three apples . A quarter pound . It took us many years of work to get a reliable symbolic way of displaying products, with which we can immediately calculate the nutritional properties and much more.



Another long-awaited novelty - historical data by country. We had data on countries in recent times (as a rule, since the 1960s or 1970s). What about an earlier period? What about Prussia ? What about the Roman Empire ? In general, in the 11th version, we finally got at least approximate information about the borders of all serious countries throughout history. Thus, through Wolfram Language, calculations can be made regarding the rise and fall of empires. Speaking of history, I want to note a small but very useful addition to the 11th version: historical data on the frequency of use of words. Just get the time series using the WordFrequencyData function



frequency of use of words, and you will find out how many people spoke about the word “war” (or “turnips”) at different times. Almost every chart is a history lesson. Another convenient feature from the 11th version is WikipediaData , which, upon request, issues any Wikipedia article (or the various types of data that it contains). There is also a WolframLanguageData function that allows you to get calculated data about the Wolfram language itself: examples shown in the documentation, relationships between functions, and so on. In many areas, we are mainly dealing with static data (" What is the density of gold? "; " What was the population of London in 1959?





"). But there are other areas in which there is little interest in static data. There are several new examples in the 11th version. For example, mortality data (“ What is the probability of dying between age X and age Y? ”), standard data on the oceans (“ What is the pressure at depth X? ”), data on the braking energy of radioactive particles in matter (radioactive stopping power) and data on human growth - as well as information about the entire Universe according to the standard cosmological model.

In addition, in 11 -th version features such as WeatherForecastData (p weather forecasting anywhere in the world) andMathematicalFunctionData (a database of mathematical formulas containing hundreds of thousands of mathematical formulas, representations of functions, etc.). Oh, as well as data on Pokemon and many other useful things.

Calculations with real objects

One of the most powerful features of Wolfram Language is its ability to perform direct calculations with real objects. The USA, or Russia, or even a lizard, for Wolfram Language are just objects that can be manipulated as symbolic constructs using the common symbolic paradigm of the language. The objects themselves do not matter; they are just symbols. But their properties may matter, say, computing the “Population” property of entity [[USA]] will yield 322 million.

However, let's assume that we want to not just take some object (for example, the USA) and find the value of its properties. Say instead we want to find objects with specific properties and values. Let's say we want to find the 5 largest countries in the world in terms of population. In the 11th version, we can do it in a new way. Instead of specifying a specific explicit object, we specify a calculation that implicitly defines the class of objects. So, for example, we can get a list of the 5 largest countries in terms of population like this: TakeLargest [5] is the operator form of a new function that works with the largest elements in the list. Implicit objects end up often using operator forms - as in Dataset queries



. And in a way, they use the symbolic character of the Wolfram Language because they work with functions that define them as data.

The mechanism of objects and properties and implicit objects works for all the different types of objects that exist in the Wolfram language. However, starting with version 11, it is not limited to built-in object types. The new design, called EntityStore , allows you to define your own types of objects, as well as specify their properties and values, and so on, and then use them in any calculations.

Just as Dataset is a powerful hierarchical generalization of typical database concepts, EntityStoreis a kind of symbolic generalization of a typical relational database. And if you created a complex EntityStore object, you can simply use the CloudDeploy function  to deploy it in the cloud and use it whenever you want.

Advanced geographic computing and visualization capabilities

Geography is one aspect of "knowledge of the real world." But Wolfram Language does not just have access to detailed geographic data (for Earth, Moon , Mars, and even Pluto ); he can also carry out calculations with this data. In his arsenal - a colossal collection of instantly computable geo-projections and all the settings to support detailed geodesy . Remember spherical trigonometry ? Wolfram Language not only knows that the Earth is a sphere, but also, using the actual shape of the Earth, correctly calculates distances, builds areas and so on on a map with arbitrary projections. If we talk about creating maps



, then Wolfram Language now has access not only to a map of streets around the world, but also to such things as the historical borders of countries , as well as satellite images , with low resolution (at least). Given street maps, a new important class of possible calculations appears: tourist destinations (and travel times ) from anywhere and anywhere.

Do not forget about the complex tasks of mathematical analysis and theoretical physics ...

In version 11, there are many new features in all areas of Wolfram Language . However, they are now available in areas traditional for Mathematica, for example, mathematical analysis .

For example, there are functions (like DEigenvalues ) for working with eigenvalues ​​and functions of differential operators for ordinary differential equations and partial differential equations . To make this possible, a huge stack of algorithmic technologies is needed, and we have been moving in this direction for more than 25 years. It is important that this is not the case when it is necessary to pose a particular problem with the help of detailed knowledge of numerical analysis. Here you define onlyequations and their boundary conditions - and the system automatically finds out how to solve them. Back in 1976 I wrote a program in Fortran , designed to search for eigenfunctions of the Schrödinger equation for the physics of elementary particles , which I studied. In 1981, I wrote C programs to do the same for some equations of relativistic quantum mechanics . I patiently waited for the day when I could just drive these tasks and get answers right away. And now it has become possible.





Of course, in the 11th version, all this is much more general. I was dealing with simple boundary conditions; but in the 11th version, in order to adjust the boundary conditions, you can use the entire geometric sub-language of Wolfram Language - and all the data that is available to us. So to find your own surface "drum" of any shape - for example, in the shape of the United States - is not difficult. In this case, we have no choice but to do all the calculations numerically. However, in the 11th version it will soon become possible to search for the eigenvalues ​​and functions of differential operators analytically. The 11th version also added some new features to solve



in the general form of (analytically) partial differential equations . In particular, we had a large R&D project, which now led us to the point where we can find a symbolic solution for almost any partial differential equation that may appear in any scientific work or textbook (if it has an analytical solution at all )

Back in 1979, when I created the predecessor of the Mathematica system (see the post on the Habré " A Brief History of the emergence of Mathematica "), I made a list of things that, I hoped, we would eventually be able to implement. One of the items on this list is the solution of integral equations. Well - glad to announce that 37 years later, we finally gota stack of algorithmic technologies to make this possible, and the 11th version presents solutions to many classes of integro-differential equations .

There is one more thing - for example, the construction of Green's functions for differential equations. And finally, the long-awaited (at least for me) function of the Mellin integral transform (it has been my favorite since it became the center of my article on particle physics in 1977).

We have put a lot of effort into what can be called " modern matanalysis " - those aspects that are necessary to support areas like machine learning. We now have more efficient and reliableminimization methods, as well as complex Bayesian minimization , designed for uncontrolled machine learning .

Education

Partial differential equations are complex from a mathematical point of view, which is reflected in their application in practice in physics and technology and so on. But what about basic math corresponding to the level of school education? Well - for a long time, Mathematica completely covered this level. But, as our stack of algorithmic technologies expanded, even for more elementary mathematics some new things became possible.

As one example, automatic processing of discontinuities, asymptotes, etc., on function graphs can be given . So now, for example, the function Tan [x] is constructed in an ideal "school" way, without gaps from –∞ to + ∞. In the case of the Tan [x] functioneverything is pretty simple. However, there are some serious algorithmic technologies inside that are designed to work with much more complex functions. And, by the way, in the new version there is something even more ambitious - this is the function MathematicalFunctionData : access to hundreds of thousands of relations and properties of mathematical functions - in a sense, these are encapsulations of many centuries of mathematical research (see the post on Habr " Computed knowledge and the future of pure mathematics ") with the goal of making them easily accessible for computation.





Recently, we have been using Wolfram Language as a way to teach algorithmic thinking at all levels. And, among other things, we wanted to make sure that any mathematical computation encountered in elementary school can really be done elementarily using the Wolfram Language. Therefore, we have such small functions as NumberExpand - it takes a number, say 123, and paints it in the form {100, 20, 3} ; we also added features such as RomanNumeral , and others .

And - partly as a tribute to the legacy of the Logo language - we created the AnglePath function - a kind of industrial scale “turtle graphics ”, which can be useful not only for primary education, but also for serious modeling - for example, random walks.

Combining all the functions into one

One of the main goals of Wolfram Language is to make everything work together seamlessly. And in the 11th version there are some new examples of its achievement.

Time series , for example, currently work directly with arithmetic . So you can take two time series, say, air pressure, and then just subtract them. Of course, it would be easier if all the time points in the series were lined up, but this is not necessary in the 11th version: Wolfram Language automatically processes irregular time series.

Another example concerns units. In the new version, statistical distributions today work seamlessly with units. Thus, the normal distribution can have not only a dispersion of 2.5 , but also a dispersion of 2.5 meters . All calculations, and the conversion of units of measurement are processed fully automatically.

Geometry has also been integrated into larger parts of the system. Those calculators that previously simply took variables can now operate on arbitrary areas. Another new feature, ImageMesh , connects images to areas: creates a geometric grid from any image. So, for example, if you want, you can make serious geometric calculations of your favorite picture with a cat.

Another example: random objects. Randomintegerand RandomReal are old features. In version 8, the RandomVariate function was introduced in order to create a set of numbers that have a given probability distribution. Then, in the 9th version , the RandomFunction function appeared , designed to generate functions from random processes. The 11th version has even more functions for working with randomness - I'm talking about the function RandomPoint , which creates a set of points randomly distributed in any area (one-, two-, three-dimensional and multi-dimensional). Also there was a function RandomEntity , which selects a random object from the built-in database of curated data, as well as a functionRandomWord , necessary for natural language research; and it’s also a good way to check your vocabulary - you can choose words from a large list of languages ... And finally, in the 11th version there was an important new area of random matrices - implemented with all the depth and completeness, which we made the hallmark of Mathematica and Wolfram Language

Visualization

One of the major achievements of Mathematica and the Wolfram language was the inclusion of visualization in everyday work. Our goal has always been to automate and visualize as much as possible. And in the 11th version, a whole collection of new automatic visualization functions is now presented .

There is a very flexible WordCloud function for building word clouds that allow you to visualize text and lines. There is a TimelinePlot function for visualizing events in time. There are AudioPlot that can render audio of various lengths. There is a Dendrogram function that uses machine learning methods to demonstrate hierarchical clusteringimages, text, or any other kind of data. There is a GeoHistogram for building geographic histograms. The TextStructure function graphically depicts a grammar of English sentences. And there is the AnatomyPlot3D function , demonstrating the elements of the human body (using symbolic specifications, since there are no explicit coordinates). What else can be visualized? There is one thing that I have visualized a lot (especially in my work in the field of basic science ) - these are the rules for simple programs like cellular automata, Turing machines, substitution rules, etc. And in the 11th version we added the RulePlot function for automatic visualization of rules in a variety of styles.







Another long-standing visualization problem was the automation of 3D data distribution . The main difficulty is to “see” the volume in 3D. But in the new version, we now have a bunch of functions that solve this in different ways, often - making slices in places that are determined by our system of calculated geometry . Another important area of ​​automation in the field of visualization is marking. The 11th version of the function is available the Callout , indicating balloons for points, lines and areas (we already have a function signature , tips and so on



) There is such a way to make footnotes - always put them, for example, on the left. However, in practice this is not very convenient, because they begin to get confused sooner or later. Instead, in the 11th version, we did something much more complex, including the algorithmic layout of footnotes to achieve aesthetic and communication goals.

From lines to text

Using Mathematica and Wolfram Language, you could always process character strings . In the 10th version, we took a huge step forward with the introduction of the Interpreter function : we used our achievements in understanding the natural language and applied them to the interpretation of strings in hundreds of different areas. Well, in the 11th version we went even further, providing a variety of functions for large-scale processing of the natural language and work with text .

There are TextWords and TextSentences functions for breaking text into words and sentences (this will avoid embarrassment like mistakes in writing “St. John” (in the original “St. John's St.”)). There are also features like TextCasesthat allow you to automatically select different classes of the natural language, such as countries, or dates, or even nouns or verbs.

It is quite interesting to consider words as data: for example, the WordList function gives lists of various kinds of words, and WordDefinition gives definitions.

There are also multilingual features . Thanks to the Alphabet function , almost any alphabet is available; Transliterate transliterates between different writing systems. And finally, the WordTranslation function can translate words into 200 languages ​​- excellent material for any linguistic research.

Modern approach to system programming

Wolfram Language is an ultra-high level language. However, in the 11th version, we added a bunch of features to cover low-level computer systems. The first is the ByteArray function , which can store and process raw byte sequences. There are also functions that deal with networks - these are PingTime and SocketConnect .

A new framework for publishing-subscribing channels has also appeared . You can create a channel, and then Wolfram Language or an external system will send data to it, and you can create a “receiver” that will carry out some actions through the Wolfram Language when the data arrives. There is much more to be done with this setting: whether it’s connecting toexternal services or devices , processing e-mail or third-party authentication, or even creating your own chat .

Another new feature is integrated cryptography . This is an exclusively symbolic framework that allows you to configure almost any protocol using public or private key systems.

But what about interacting with the web? The symbolic character of the Wolfram Language is also strongly manifested here. For example, it allows you to interact with HTTPRequest and HTTPResponse as with character structures . There is also a URLSubmit functionto work with character-specific handler functions for feedback from asynchronous URL execution. There is even a CookieFunction function responsible for the symbolic processing of cookies.

You can do system programming in almost any language, or even, for example, on the command line. However, I found that doing this in Wolfram Language is much more effective. Let's say you research the performance of a computer system (see the post on Habr " Search for errors in the cloud from a scientific point of view: the unexpected adventure of CEO "). Everything you do is beautifully stored in the documentwhere you can add comments, etc. And then everything that you do can be immediately visualized. Or you can use machine learning, or anything else. Want to test network performance? Use the PingTime function to create a list of delays in data transfer; then make a histogram correlating with other data.

The FileSystemMap function , another novelty, is able to process the file system as a set of nested lists (or associations), and then apply a function on it. So, for example, you can take the whole catalog of images and use FileSystemMap , applying the image processing functions for all images at once.

Yes, and one more thing: the 11th version also includes (although it is still marked as experimental), industrial-level systems for searching for text documents both locally and in the cloud.

Internet work

An incredible feature of the Wolfram language is that it works not only on the desktop, but also in the cloud . And in the 11th version, we added many new cloud-based features for working on the Internet.

Let's start simple. Function CloudDeploy [ FormFunction [...]] allows you to instantly create an application on the basis of the forms directly to the Internet. But now you can make the form even more complex. There are many new “smart fields” that automatically use the natural understanding of the language to interpret the entered data. There are also new constructs like RepeatingElement and CompoundElementthat automatically set the fields for retrieving data for lists and associations. And there is a new programmable linguistic interface that allows you to define your own grammar in order to expand your understanding of the natural language already built into the Wolfram Language.

Those forms that you symbolically designate in Wolfram Language can be quite complex - over several pages with a lot of interdependencies and formatting. But ultimately they remain the same forms for which you set your input data and then send them. The 11th version introduces a new framework - AskFunction, which allows you to create more complex interactions: for example, dialog boxes in which you “interview” the user to obtain data. In Wolfram Language, the entire process is defined by a character structure, which then immediately becomes active on the Internet using the CloudDeploy function .

Wolfram Language's goal is to be able to easily do complex things on the Internet. In the 11th version, we included the FormPage function (as in wolframalpha.com ), as well as GalleryView , which allows you to create a list of images in Wolfram Language, and then expand it on the Internet as a “gallery” (as in demonstrations.wolfram.com ).



If you want to work at a lower level, there are URLDispatcher and GenerateHTTPResponse functions for this , which allow you to determine exactly how web requests will be processed depending on your cloud settings.

In addition, the new version has features such as CloudPublish and CloudShare that allow you to control access to what you have placed in the cloud. Another small but important feature is SourceLink., which allows you to automatically associate, say, a graphic image that you place in the cloud with the document (also in the cloud) in which it was created. I think this is a great tool with which every image you see in a document contains a link to the person who created it. I insist that inside our company, automated reports are written in Wolfram Language, and, of course, include source links, so that I can always get the source data and analyze it myself.

Cloud data

You could use Wolfram Data Drop already in the 10th version, but now an extended version is available that allows you to accumulate data in the Wolfram Cloud . I must say that I underestimated the capabilities of Wolfram Data Drop . I thought (see a post on the Habr " Wolfram Data Drop - the new Wolfram Research service ") that it will be used primarily for storing data from sensors, etc. . And, indeed, many applications are thought out in these areas. However, for Data Drop, you can find use exclusively within the Wolfram Language. Say you have a web form that runs on Wolfram Language. You can process each request and then cast the result in the formatDatabin in Wolfram Data Drop to analyze everything at once.

Wolfram Data Drop is designed primarily to accumulate time series data. In the 11th version, there was another way to store data in the cloud - CloudExpression . You can put any Wolfram Language expression in the cloud, and it will be stored there, and each part of it can be retrieved or configured using ordinary functions (for example, Part or AppendTo ). CloudExpression is a great way to store structured data in which details are constantly changing, but the data itself remains unchanged in the cloud.

What you put in the cloud remains unchanged. In the 11th version, a function also appearedLocalObject is a local analogue of the CloudObject function, which provides permanent local storage on your computer, and LocalCache , which is used to cache objects in local storage.

Connect to any external services: Facebook, Twitter, Instagram, ArXiv, Reddit and many others ...

With the help of Wolfram Language we curate a lot of data that we put directly into the Knowledge Base ; we are also looking for ways to further obtain data - such as external APIs . In the 11th version, many new connections appeared - with Flickr , Reddit , MailChimp , SurveyMonkey , SeatGeek , ArXiv and so on . Wolfram Language is a powerful tool for deploying your own APIs. And in the new version there are a number of authentication mechanisms that are supported for the API: for example, PermissionsKey for assigning AppID. CloudLoggingData



provides the most detailed data on how access to any API or any other cloud object is provided.

The API through which you make calls to various services on the Internet receives data transmitted to it through the specified URL. In the 11th version, a new kind of API is now available that works through email, and not through the Internet and URLs. The MailReceiverFunction function is similar to APIFunction except that it first determines the email address, and then any mail sent to this address is passed to the code of the MailReceiverFunction function . MailReceiverFunctionallows you to separate the different parts of the mail messages and their headers, and then using the Wolfram Language to automate the processing of e-mail of almost any level of complexity. And for people like me who receive a huge amount of emails from both people and automated systems, it's pretty convenient.

WolframScript

You can access the Wolfram language through a laptop, desktop or cloud. You can access it using deferred tasks in the cloud or through the API or the MailReceiverFunction function . Wolfram Language could always be run from the command line, but in the new version it became possible to do this using WolframScript .

The idea behind WolframScript is to create a very simple but flexible interface to Wolfram Language. WolframScript allows you to run Wolfram Engine on your computer , which in turn makes it possible to run code from a file or directly from the command line. This allows you to receive feedback in any format.- including text, images, sounds, PDF, CDF , etc. If you are dealing with Unix, then you can use #! Wolframscript to create a script that can be called stand-alone and that will work with WolframScript.

There is one more thing. You can customize how WolframScript works like FormFunction to pull out the arguments of any type you designate (and interpret when necessary). And you can also use WolframScript to call the API you already created in the cloud.

In our company, we often use the Wolfram Language as part of a large and distributed system. WolframScript provides a direct way to place components in Wolfram Language.

New at the core of the Wolfram Language

I have already talked about all kinds of innovations that expand and deepen the algorithmic capabilities of Mathematica and Wolfram Language. But what about the core structure of Wolfram Language itself ? Of course, we strive to maintain compatibility (although this is not difficult, given the fact that we are always focused on development). But we also want to gradually strengthen and polish the language.

The process of evolution of natural languages ​​involves, among other things, the construction of new words from idioms. Essentially, we do the same in Wolfram Language. We study which clusters of recalculations most often appear in Wolfram Language code. Then we come up with a good name for this particular set of calculations and create a new function.

At the beginning of my work on the Mathematica system, I thought that it would be cool if there were functions that allow you to do something using idioms. I realized that if an idiom is compressed to one function, from the name of which it is clear why it is needed, then we get a code that is easier to read. And the fact that idioms do not have to be reconstructed many times justifies the introduction of a new function.

Over the past few years, our company is actively developing two areas: Language Development (Incremental Language Development - ILD), and Consistency & Completeness language (Language Consistency & Completeness - LCC) . The main idea of ​​ILD is the introduction of functions equivalent to idioms. The main idea of ​​LCC is to make sure that both template expressions andunits of measure and character URLs are supported wherever it makes sense.

So, for example, the standard addition of ILD in the 11th version is the MinMax function , which returns the minimum and maximum values ​​for a given list (it is surprising that it saves from how many difficulties when using Map ). Standard addition LCC - support the search for a pattern in the associations .

In the 11th version, many additions to the main language appeared . Functions like Cases were common and SequenceCases appeared - searching for sequences instead of individual items in a list. There is still a functionA SequenceFoldList that is similar to a FoldList except that it can return to a sequence of elements of any length. There is also a similar function FoldPairList , which generalizes the FoldList and returns at each step a result that differs from that achieved earlier. This may seem abstract, but it is a very useful operation when you want to maintain a separate internal state while continuing to receive data.

Another novelty that may seem strange at first glance is the Nothing function. Despite its name, Nothingdoes something: whenever it appears in the list, it is immediately deleted. This means that if you want to get rid of the list item, you must replace it with Nothing .

Thanks to some innovations, using the 11th version has become more convenient. For example, for the First function there is now a second argument, which says what to do if there is no first element, and eliminates the need to include If in this case . The UpTo function is designed to display a certain number of objects - you can specify: Take [ list , UpTo [4]] to get up to 4 list elements, (if there are fewer, then as many as there will be). UpTo is supported in many places, and this greatly simplifies the code.

Echo is another convenient feature. When you try to track what is going on inside a piece of code, you may need to print some intermediate result. Echo is a function that prints and then returns printed, so you can use it in your code without changing what this code does.

It is hard to believe that you can add anything else to this list, but that is not all. Subdivide is a function similar to Range except that it divides the range into equal parts. TakeLargest and related functions summarize functionsMax and the like, to take not only the largest, but n largest list items.

There is also a Groupings function, the structure of which I have been thinking over for many years: it generates all possible trees formed by binary or other adders ("What numbers can you get by putting Plus signsand Times multiplicationsin the list of units in all possible ways?").

The Table function isconvenient in that you can now specify Table [ x , n ] rather than Table [ x , { n }]. In general, there are many more things that make the core of the 11th version of Wolfram Language more elegant and convenient to use.

And much more ...

It was a long post. However, I did not even closely describe everything new that appeared in the 11th version. There is more information on the Internet. Check out the New in Wolfram Language 11 page , or view user information or a brief description of new features. See also the list of new features starting from version 10.4 up to 11.0 .

But most importantly - start using the new version! If you want to quickly (and for free ) interact with it, launch it through Wolfram Open Cloud . Or just start using Mathematica 11 or the 11th version of any other Wolfram Language- based products .

For some time now I have been using test versions, and the 10th version for me looks and feels very “old-fashioned” - I miss these wonderful new interface features, new functionalities and small amenities. I am very pleased with how the 11th version turned out. This is another big step on our 30-year development path for Mathematica and Wolfram Language. And I'm glad that the most diverse people around the world will be able to do amazing things for the first time with Mathematica 11 and other products of the 11th version of Wolfram Language .

For questions about Wolfram technologies, write to info-russia@wolfram.com