May 22, 2019 at 16:16JMAP - an open protocol replaces IMAP when exchanging emails
Earlier this month , the JMAP protocol, developed under the direction of the IETF , was actively discussed at Hacker News . We decided to talk about why it was needed and how it works. / PxHere / PD
Protocol IMAP was introduced in 1986. Many of the things described in the standard are no longer relevant today. For example, the protocol can return the number of message lines and MD5 checksums - this functionality is practically not used in modern mail clients.
Another problem is traffic consumption. In the case of IMAP, emails are stored on the server and periodically synchronized with local clients. If for some reason the copy on the user's device "spoils", you have to re-synchronize all mail. In the modern world, when thousands of mobile devices can be connected to the server, this approach leads to increased consumption of traffic and computing resources.
Difficulties arise not only with the protocol itself, but also with the email clients that work with it. Since its inception, IMAP has undergone various revisions many times - the current version is IMAP4 today. Moreover, there are many optional extensions for him - ninety RFCs with additions are published on the network . One of the freshest is RFC8514 , introduced in 2019.
However, many companies offer their own proprietary solutions that should simplify the work with IMAP or even replace it: Gmail , Outlook , Nylas. The result is that existing email clients support only a few of all the available features. Such diversity leads to market segmentation.
JMAP is called to solve the designated problems. It is developed by experts from FastMail under the leadership of the Internet Engineering Council (IETF). The protocol runs on top of HTTPS, uses JSON (for this reason, it is suitable not only for electronic messaging, but also for solving a number of tasks in the cloud) and simplifies the organization of work with mail in mobile systems. In addition to processing letters, JMAP also provides the ability to connect extensions for working with contacts and the calendar planner.
JMAP is a stateless protocol and does not require a permanent connection to the mail server. This feature simplifies work in unstable mobile networks and saves battery power devices.
The email in JMAP is presented in the format of a JSON structure. It contains all the information from the RFC5322 message (Internet Message Format) that mail applications may need. According to the developers, this approach should simplify the creation of clients, since the server will be responsible for solving potential difficulties (related to MIME , reading headers and encoding).
The client uses the API to access the server. To do this, it generates an authenticated POST request, the properties of which are described in the JMAP session object. The request is in application / json format and consists of a single JSON request object. The server also generates a single response object.
In the specification (paragraph 3), the authors give the following example with a request:
Below is an example of the response that the server will form:
The full JMAP specification with implementation examples can be found on the project's official website . In the same place, the authors posted a description of the specifications for JMAP Contacts and JMAP Calendars - they are aimed at working with calendars and contact lists. According to the authors, Contacts and Calendars were separated into separate documents so that they could be further developed and standardized regardless of the “core”. The source code for JMAP is in the repository on GitHub .
/ PxHere / PD
Despite the fact that officially the work on the standard has not yet been completed, it is already being implemented in production environments. For example, the creators of the Cyrus IMAP open mail server implemented its JMAP version. FastMail developers released a server framework for the new Perl protocol, while JMAP authors introduced a proxy server .
It can be expected that in the future there will be more and more JMAP-based projects. For example, there is some likelihood that developers from Open-Xchange will create a new protocol and create an IMAP server for Linux systems. To refuse IMAP they are very much asked by members of the community that has formed around the company's tools.
IETF and FastMail developers say more and more users see the need for a new open standard for messaging. The authors of JMAP hope that in the future more companies will begin to implement this protocol.
What did not please IMAP
Protocol IMAP was introduced in 1986. Many of the things described in the standard are no longer relevant today. For example, the protocol can return the number of message lines and MD5 checksums - this functionality is practically not used in modern mail clients.
Another problem is traffic consumption. In the case of IMAP, emails are stored on the server and periodically synchronized with local clients. If for some reason the copy on the user's device "spoils", you have to re-synchronize all mail. In the modern world, when thousands of mobile devices can be connected to the server, this approach leads to increased consumption of traffic and computing resources.
Difficulties arise not only with the protocol itself, but also with the email clients that work with it. Since its inception, IMAP has undergone various revisions many times - the current version is IMAP4 today. Moreover, there are many optional extensions for him - ninety RFCs with additions are published on the network . One of the freshest is RFC8514 , introduced in 2019.
However, many companies offer their own proprietary solutions that should simplify the work with IMAP or even replace it: Gmail , Outlook , Nylas. The result is that existing email clients support only a few of all the available features. Such diversity leads to market segmentation.
“Moreover, a modern email client should not only forward messages, but be able to work with contacts and synchronize with the calendar,” says Sergey Belkin, head of development for IaaS provider 1cloud.ru . - Today, third-party protocols like LDAP , CardDAV and CalDAV are used for these purposes . This approach complicates the configuration of firewalls in corporate networks and opens up new vectors for cyber attacks. ”
JMAP is called to solve the designated problems. It is developed by experts from FastMail under the leadership of the Internet Engineering Council (IETF). The protocol runs on top of HTTPS, uses JSON (for this reason, it is suitable not only for electronic messaging, but also for solving a number of tasks in the cloud) and simplifies the organization of work with mail in mobile systems. In addition to processing letters, JMAP also provides the ability to connect extensions for working with contacts and the calendar planner.
Features of the new protocol
JMAP is a stateless protocol and does not require a permanent connection to the mail server. This feature simplifies work in unstable mobile networks and saves battery power devices.
The email in JMAP is presented in the format of a JSON structure. It contains all the information from the RFC5322 message (Internet Message Format) that mail applications may need. According to the developers, this approach should simplify the creation of clients, since the server will be responsible for solving potential difficulties (related to MIME , reading headers and encoding).
The client uses the API to access the server. To do this, it generates an authenticated POST request, the properties of which are described in the JMAP session object. The request is in application / json format and consists of a single JSON request object. The server also generates a single response object.
In the specification (paragraph 3), the authors give the following example with a request:
{
"using": [ "urn:ietf:params:jmap:core", "urn:ietf:params:jmap:mail" ],
"methodCalls": [
[ "method1", {
"arg1": "arg1data",
"arg2": "arg2data"
}, "c1" ],
[ "method2", {
"arg1": "arg1data"
}, "c2" ],
[ "method3", {}, "c3" ]
]
}
Below is an example of the response that the server will form:
{
"methodResponses": [
[ "method1", {
"arg1": 3,
"arg2": "foo"
}, "c1" ],
[ "method2", {
"isBlah": true
}, "c2" ],
[ "anotherResponseFromMethod2", {
"data": 10,
"yetmoredata": "Hello"
}, "c2"],
[ "error", {
"type":"unknownMethod"
}, "c3" ]
],
"sessionState": "75128aab4b1b"
}
The full JMAP specification with implementation examples can be found on the project's official website . In the same place, the authors posted a description of the specifications for JMAP Contacts and JMAP Calendars - they are aimed at working with calendars and contact lists. According to the authors, Contacts and Calendars were separated into separate documents so that they could be further developed and standardized regardless of the “core”. The source code for JMAP is in the repository on GitHub .
/ PxHere / PD
Prospects
Despite the fact that officially the work on the standard has not yet been completed, it is already being implemented in production environments. For example, the creators of the Cyrus IMAP open mail server implemented its JMAP version. FastMail developers released a server framework for the new Perl protocol, while JMAP authors introduced a proxy server .
It can be expected that in the future there will be more and more JMAP-based projects. For example, there is some likelihood that developers from Open-Xchange will create a new protocol and create an IMAP server for Linux systems. To refuse IMAP they are very much asked by members of the community that has formed around the company's tools.
IETF and FastMail developers say more and more users see the need for a new open standard for messaging. The authors of JMAP hope that in the future more companies will begin to implement this protocol.
Our additional resources and sources:How to check cookies for GDPR compliance - a new open tool will help
How to save money using the
Potential attacks on HTTPS and ways to protect against them