Yet another P2P Messenger

Reading reviews and documentation about the language is not enough to learn how to write less useful applications.

It is imperative that you need to create something interesting to consolidate, so that you can use it in your other tasks.

The article is aimed at beginners interested in the go language and peer-to-peer networks.
And for professionals who can offer intelligent ideas or constructively criticize.

I program for a long time with a different degree of immersion in java, php, js, python.
And each programming language is good in its field.

The main area for Go is the creation of distributed services, microservices.
Most often, microservice is a small program that performs its highly specialized functionality.

But microservices should still be able to communicate with each other, so the tool for creating microservices should allow you to easily and easily organize network interaction.
To check this, we will write an application that organizes a decentralized network of equal participants (Peer-To-Peer), the simplest is the p2p messenger (by the way, is there a Russian synonym for this word?).

In the code I actively invent bicycles and step on a rake in order to feel the golang, to receive constructive criticism and rational suggestions.

What we do

Feast (peer) - a unique copy of the messenger.

Our messenger should be able to:

• Find nearby peers
• Connect with other peers
• Encrypt data exchange with peers
• Receive messages from user
• Show messages to user

To make the puzzle a little more interesting, let's make it all go through one network port.

If you pull this port via HTTP, you will get a Reakt application that pulls the same port by establishing a web socket connection.

If you pull the port over HTTP from the local machine, then we show the banner.

If another peer is connected to this port, then a permanent connection is established with end-to-end encryption.

Determine the type of incoming connection

To begin, open the port to listen and wait for new connections.

net.ListenTCP("tcp", tcpAddr)

On the new connection we read the first 4 bytes.

We take a list of HTTP verbs and compare our 4 bytes with it.

Now we determine whether the connection is made from the local machine, and if not, then we answer with a banner and "hang up".

    buf, err := readWriter.Peek(4)
    /* обработка ошибки */if ItIsHttp(buf) {
        handleHttp(readWriter, conn, p)
    } else {
        peer := proto.NewPeer(conn)
        p.HandleProto(readWriter, peer)
    }
    /* ... */if !strings.EqualFold(s, "127") && !strings.EqualFold(s, "[::") {
        response.Body = ioutil.NopCloser(strings.NewReader("Peer To Peer Messenger. see https://github.com/easmith/p2p-messenger"))
    }

If the connection is local, then we respond with a file corresponding to the request.

Here I decided to write the processing myself, although I could use the handler available in the standard library.

// свой способ funcprocessRequest(request *http.Request, response *http.Response) {/* много строчек кода */}
    // либо из страндартной библиотеки
    fileServer := http.FileServer(http.Dir("./front/build/"))
    fileServer.ServeHTTP(NewMyWriter(conn), request)

If the path is requested /ws, then try to establish a websocket connection.

Since I’ve put the bike together to handle file requests, I’ll do the handling of the ws connection using the gorilla / websocket library .

To do this, create MyWriterand implement methods in it to match the interfaces http.ResponseWriterand http.Hijacker.

// w - MyWriterfunchandleWs(w http.ResponseWriter, r *http.Request, p *proto.Proto) {
        c, err := upgrader.Upgrade(w, r, w.Header())
        /* теперь работаем с соединением почти как с обычным сокетом */
    }

Peer Detection

To search for peers in a local network, we use multicast UDP.

We will send to Multicast IP address packets with information about ourselves.

funcstartMeow(address string, p *proto.Proto) {
            conn, err := net.DialUDP("udp", nil, addr)
            /* ... */for {
                _, err := conn.Write([]byte(fmt.Sprintf("meow:%v:%v", hex.EncodeToString(p.PubKey), p.Port)))
                /* ... */
                time.Sleep(1 * time.Second)
            }
    }

And separately listen to all UDP packets from Multicast IP.

funclistenMeow(address string, p *proto.Proto, handler func(p *proto.Proto, peerAddress string)) {
        /* ... */
        conn, err := net.ListenMulticastUDP("udp", nil, addr)
        /* ... */
        _, src, err := conn.ReadFromUDP(buffer)
        /* ... */// connectToPeer
        handler(p, peerAddress)
    }

Thus, we declare ourselves and learn about the appearance of other peers.

One could organize this at the IP level and even in the official documentation of the IPv4 package, just the multicast data packet is given as an example of the code.

Peer interaction protocol

We will pack all communication between peers in an envelope.

On any envelope there is always a sender and recipient, to which we add a command (which it carries with it), an identifier (as long as it is a random number, but can be made as a hash of content), the length of the content and the contents of the envelope itself - the message or command parameters.

The command, (or the type of content) is conveniently located at the very beginning of the envelope and we define a list of commands of 4 bytes that do not intersect with the names of HTTP verbs.

The entire envelope, when transmitted, is serialized into an array of bytes.

Handshake

When the connection is established, the peer immediately extends a hand for a handshake, communicating its name, public key and ephemeral public key to generate a common session key.

In response, the peer receives a similar set of data, registers the found peer in its list and calculates (CalcSharedSecret) the common session key.

funchandShake(p *proto.Proto, conn net.Conn) *proto.Peer {
        /* ... */ 
        peer := proto.NewPeer(conn)
        /* Отправляем свое имя и ключ*/
        p.SendName(peer)
        /* Ждем имя и ключ */
        envelope, err := proto.ReadEnvelope(bufio.NewReader(conn))
        /* ... */
    }

Peer Exchange

After a handshake, peers exchange their lists of peers =)

For this, an envelope is sent with the LIST command, and a JSON list of peers is put in its contents.
In response, we get a similar envelope.

We find in the lists of new ones and with each of them we make an attempt to connect, shake hands, exchange peers and so on ...

User Message Exchange

User messages are the most valuable for us, so we will encrypt and sign each connection.

About encryption

In the standard (google) golang libraries from the crypto package, there are many different algorithms implemented (no GOSTs).

I find the Ed25519 curve as the most convenient for signatures. We will use the ed25519 library to sign messages.

At the very beginning, I was thinking of using a couple of keys obtained from ed25519, not only for signing, but also for generating a session key.

However, the keys for the signature are not applicable to the calculation of the common (shared) key - they still need to be conjured over them:

funcCreateKeyExchangePair()(publicKey [32]byte, privateKey [32]byte) {
    pub, priv, err := ed25519.GenerateKey(nil)
    /* ... */copy(publicKey[:], pub[:])
    copy(privateKey[:], priv[:])
    curve25519.ScalarBaseMult(&publicKey, &privateKey)
   /* ... */
}

Therefore, it was decided to generate ephemeral keys, and generally speaking, this is the right approach, which leaves no chance for attackers to pick up the common key.

For mathematics lovers, here are the links to the wiki: Diffie
Protocol —_Hellman_a_elliptic_Digital
EdDSA Digital Signature

The generation of a shared key is quite standard: first, for the generation of a new connection, ephemeral keys, we send an envelope with a public key to the socket.

The opposite side does the same, but in a different order: it receives an envelope with a public key, generates its pair and sends the public key to the socket.

Now each participant has someone else’s public and private ephemeral keys.

Multiplying them we get the same key for both, which we will use to encrypt messages.

//CalcSharedSecret Calculate shared secretfuncCalcSharedSecret(publicKey []byte, privateKey []byte)(secret [32]byte) {
    var pubKey [32]bytevar privKey [32]bytecopy(pubKey[:], publicKey[:])
    copy(privKey[:], privateKey[:])
    curve25519.ScalarMult(&secret, &privKey, &pubKey)
    return
}

We will encrypt messages by a piece by the long-proven AES algorithm in block coupling mode (CBC).

All this implementations are easily found in the golang documentation.

The only refinement is auto-filling of the message with zero bytes for the multiplicity of its length to the length of the encryption block (16 bytes).

//Encrypt the messagefuncEncrypt(content []byte, key []byte) []byte {
        padding := len(content) % aes.BlockSize
        if padding != 0 {
            repeat := bytes.Repeat([]byte("\x00"), aes.BlockSize-(padding))
            content = append(content, repeat...)
        }
        /* ... */
    }
    //Decrypt encrypted messagefuncDecrypt(encrypted []byte, key []byte) []byte {
        /* ... */
        encrypted = bytes.Trim(encrypted, string([]byte("\x00")))
        return encrypted
    }

Back in 2013, I implemented AES (with a similar CBC mode) to encrypt messages in Telegram as part of a competition from Pavel Durov.

The most common Diffie-Hellman protocol was used in the telegrams to generate the ephemeral key at that time .

And in order to eliminate the load from fake connections, before each key exchange clients solved the factorization task.

GUI

We need to show a list of peers and a list of messages with them, as well as respond to new messages by increasing the counter next to the name of the feast.

Here without problems - ReactJS + websocket.

A web socket message is essentially a kind of envelope, only it does not contain ciphertexts.

All of them are "heirs" of the type WsCmdand are serialized during transmission to JSON.

//Serializable interface to detect that can to serialised to jsontype Serializable interface {
        ToJson() []byte
    }
    functoJson(v interface{}) []byte {
        json, err := json.Marshal(v)
        /* обработка err */return json
    }
    /* ... *///WsCmd WebSocket commandtype WsCmd struct {
        Cmd string`json:"cmd"`
    }
    //WsMessage WebSocket command: new Messagetype WsMessage struct {
        WsCmd
        From    string`json:"from"`
        To      string`json:"to"`
        Content string`json:"content"`
    }
    //ToJson convert to JSON bytesfunc(v WsMessage)ToJson() []byte {
        return toJson(v)
    }
    /* ... */

So, HTTP request comes to root ("/"), now to display the front we look in the “front / build” directory and give it to index.html

Well, the interface is set up, now the choice for users: run it in a browser or in a separate window - WebView.

For the latter option used zserge / webview

    e := webview.Open("Peer To Peer Messenger", fmt.Sprintf("http://localhost:%v", initParams.Port), 800, 600, false)

To build an application with it, you need to install another lib in the system.

    sudo apt install libwebkit2gtk-4.0-dev

During my thoughts on the GUI I found a lot of libraries for GTK, QT, and the console interface would look very geeky - https://github.com/jroimartin/gocui - in my opinion a very interesting idea.

Launch messenger

Install golang

Of course, you first need to install go.
To do this, I strongly recommend using the golang.org/doc/install instructions .

Simplified instructions to bash script

Download the application to GOPATH

That's the way go is, that all libraries and even your projects should be in the so-called GOPATH.

The default is $ HOME / go. Go allows you to extract the source from the public repository with a simple command:

    go get github.com/easmith/p2p-messenger

Now the $HOME/go/src/github.com/easmith/p2p-messengersource code from the master branch will appear in your directory.

Npm installation and front assembly

As I wrote above, our GUI is a web application with a front on ReactJs, so the front still needs to be built.

Nodejs + npm - here as usual.

Just in case, here is the instruction for ubuntu

Now we run the front assembly as standard

cd front
npm update
npm run build

Front ready!

Launch

Go back to the root and run the feast of our messenger.

At startup, we can specify the name of our peer, a port, a file with addresses of other peers, and a flag indicating whether to launch WebView.

By default, the $USER@$HOSTNAMEport name 35035 is used as the peer name.

So, run and chat with friends on the local network.

    go run app.go -name Snowden

Feedback on golang programming

• The most important thing I would like to note is that on go it immediately turns out to realize what I had in mind .
  Almost everything you need is in the standard library.
• However, there was a difficulty when I started a project in a directory other than GOPATH.
  For writing code used GoLand. And at first, the automatic formatting of the code with the auto-import of libraries confused.
• The IDE has a lot of code generators , which allowed us to focus on development, and not on typing code.
• You quickly get used to frequent error handling , but hand-to-face happens when you realize that for go there is a normal situation when the essence of the error is analyzed by its string representation.
  

  err != io.EOF

• Slightly better things are with the os library. They help to understand the essence of the problem.
  

  if os.IsNotExist(err) { /* ... */ }

• Out of the box go teaches us to properly document the code and write tests.
  And there is their own but. We have described the interface with the method ToJson().
  So, the documentation generator does not inherit the description of this method on the methods that implement it, so in order to remove unnecessary warnigs, you have to copy the documentation into each implemented method (proto / mtypes.go).
• Recently I got used to the power of log4j in java, so there is not enough good logger in go.
  It is probably worth looking for a beautiful logging with appenders and formatters in the githab open spaces.
• Unusual work with arrays.
  For example, concatenation occurs through a function append, and the transformation of an array of arbitrary length into an array of fixed length through copy.
• switch-caseIt works like this if-elseif-else, but this is an interesting approach, but again hand-to-face:
  if you want habitual behavior switch-case, you need to affix each case fallthrough.
  And you can still use goto, but let's not, please!
• There is no ternary operator and often this is not convenient.

What's next?

Here is the simplest Peer-To-Peer messenger.

There are bumps, you can further improve the user functionality: sending files, pictures, audio, smiles, etc., etc.

And you can not reinvent your own protocol, and use the google Protocol Buffers,
connect the blockchain and protect against spam using smart contracts Ethereum.

On smart contracts, organize group chats, channels, name system, avatars and user profiles.

It is also necessary to start seed peers, implement a NAT traversal and transfer messages from feast to feast.

In the end, you will get a good telegram / VTS replacement, all you have to do is transplant all your friends =)

Utility

godoc -HTTP=:6060