Introduction to the Go Module System

Original author: Roberto Selbach
  • Transfer

The upcoming release of version 1.11 of the Go programming language will bring experimental support for the modules — a new dependency management system for Go. (comment perev .: release took place )


Recently, I wrote about this small post . Since then, something has changed slightly, and we have become closer to release, so it seems to me that the time has come for a new article - add more practice.


So, here's what we will do: create a new package and then make a few releases to see how it works.


Module creation


First, create our package. Let's call it "testmod". An important detail: the package directory should be placed outside yours $GOPATH, because, inside it, module support is disabled by default . Go modules are the first step to complete failure in the future $GOPATH.


$ mkdir testmod
$ cd testmod

Our package is quite simple:


package testmod
import"fmt"// Hi returns a friendly greetingfuncHi(name string)string {
   return fmt.Sprintf("Hi, %s", name)
}

The package is ready, but it is not yet a module . Let's fix it.


$ go mod init github.com/robteix/testmod
go: creating newgo.mod: module github.com/robteix/testmod

We have a new file with the name go.modin the package directory with the following contents:


modulegithub.com/robteix/testmod

Not much, but this is exactly what turns our package into a module .


Now we can push this code into the repository:


$ git init 
$ git add * 
$ git commit -am "First commit" 
$ git push -u origin master

Until now, anyone willing to use our package would apply go get:


$ go get github.com/robteix/testmod

And this command would bring the latest code from the branch master. This option still works, but it would be better for us not to do this anymore, because now “there is a better way. It’s dangerous to pick up the code directly from the branch master, because we never know for sure that the package authors didn’t make any changes that would break our code. To solve this particular problem, the Go modules were invented.


A small digression about module versioning


Go modules are versioned, plus there is some specificity of individual versions. You will have to become familiar with the concepts underlying semantic versioning .


In addition, Go uses the repository tags when looking for versions, and some versions are different from the rest: for example, versions 2 and more should have a different import path than for versions 0 and 1 (we get to this).


By default, Go loads the latest version with a label available in the repository.
This is an important feature, since it can be used when working with a branch master.


What is important for us now is that when creating the release of our package, we need to put a label with the version in the repository.


Let's do it and do it.


Making your first release


Our package is ready and we can "release" it to the whole world. We do this with the help of version labels. Let the version number be 1.0.0:


$ git tag v1.0.0
$ git push --tags

These commands create a label in my Github repository, which marks the current commit as release 1.0.0.


Go does not push this, but it’s a good idea to create an additional new branch ("v1") to which we can send fixes.


$ git checkout -b v1
$ git push -u origin v1

Now we can work in a branch masterwithout worrying that we can break our release.


Using our module


Let's use the created module. We will write a simple program that imports our new package:


package main
import (
    "fmt""github.com/robteix/testmod"
)
funcmain() {
    fmt.Println(testmod.Hi("roberto"))
}

Until now, you would launch go get github.com/robteix/testmodto download a package, but with modules it becomes more interesting. First we need to include support for modules in our new program.


$ go mod init mod

As you probably expected, based on what was read earlier, a new file appeared in the directory go.modwith the module name inside:


modulemod

The situation becomes even more interesting when we try to build our program:


$ go build
go: finding github.com/robteix/testmod v1.0.0go: downloading github.com/robteix/testmod v1.0.0

As you can see, the team goautomatically found and downloaded the package imported by our program.
If we check our file go.mod, we will see that something has changed:


modulemodrequire github.com/robteix/testmod v1.0.0

And we have another new file with the name go.sum, which contains package hashes, to check the correctness of the version and files.


github.com/robteix/testmod v1.0.0 h1:9EdH0EArQ/rkpss9Tj8gUnwx3w5p0jkzJrd5tRAhxnA=
github.com/robteix/testmod v1.0.0/go.mod h1:UVhi5McON9ZLc5kl5iN2bTXlL6ylcxE9VInV71RrlO8=

Making a release with bug fix.


Now, let's say we found a problem in our package: there is no punctuation in the greeting!
Some people will get mad, because our friendly greeting is no longer so friendly.
Let's fix this and release a new version:


// Hi returns a friendly greetingfuncHi(name string)string {
-       return fmt.Sprintf("Hi, %s", name)
+       return fmt.Sprintf("Hi, %s!", name)
}

We made this change right in the branch v1, because it has nothing to do with what we will do next in the branch v2, but in real life, you might have to make these changes masterand then backport them to v1. In any case, the fix should be in the thread v1and we need to mark it as a new release.


$ git commit -m "Emphasize our friendliness" testmod.go
$ git tag v1.0.1
$ git push --tags origin v1

Modules update


By default, Go does not update modules without demand. "And that's good," because we all would like to be predictable in our builds. If the Go modules would be updated automatically every time a new version comes out, we would return to the “Dark Ages to-Go1.11”. But no, we need to tell Go to update the modules for us.


And we will do it with the help of our old friend - go get:


  • run go get -uto use the last minor or patch release (i.e., the command will update from 1.0.0 to, say, 1.0.1 or to 1.1.0, if such a version is available)


  • Run go get -u=patchto use the latest patch version (i.e., the package will be updated to 1.0.1, but not to 1.1.0)


  • run go get package@versionto upgrade to a specific version (for example github.com/robteix/testmod@v1.0.1)



In this list there is no way to upgrade to the latest major version. There is a good reason for this, as we shall soon see.


Since our program used version 1.0.0 of our package and we just created version 1.0.1, any of the following commands will update us to 1.0.1:


$ go get -u
$ go get -u=patch
$ go get github.com/robteix/testmod@v1.0.1

After launch (let's say go get -u), our go.modchanged:


modulemodrequire github.com/robteix/testmod v1.0.1

Major versions


In accordance with the specification of semantic versioning, the major version differs from the minor version . Major versions can break backward compatibility. From the point of view of the Go modules, the major version is a completely different package .


It may sound crazy at first, but it makes sense: two versions of the library that are incompatible with each other are two different libraries.


Let's make a major change in our package. Suppose, over time, it became clear to us that our API is too simple, too limited for our users к usceices ’, so we need to change the function Hi()to accept the greeting language as a parameter:


package testmod
import (
    "errors""fmt" 
) 
// Hi returns a friendly greeting in language langfuncHi(name, lang string)(string, error) {
    switch lang {
    case"en":
        return fmt.Sprintf("Hi, %s!", name), nilcase"pt":
        return fmt.Sprintf("Oi, %s!", name), nilcase"es":
        return fmt.Sprintf("¡Hola, %s!", name), nilcase"fr":
        return fmt.Sprintf("Bonjour, %s!", name), nildefault:
        return"", errors.New("unknown language")
    }
}

Existing programs using our API will break down because they a) do not pass the language as a parameter and b) do not expect an error to return. Our new API is no longer compatible with version 1.x, so meet version 2.0.0.


Earlier, I mentioned that some versions have features, and now such a case.
Versions 2 and more should change the import path. Now they are different libraries.


We will do this by adding a new version path to the name of our module.


modulegithub.com/robteix/testmod/v2

Everything else is the same: push, put a label that is v2.0.0 (and optionally sod v2)


$ git commit testmod.go -m "Change Hi to allow multilang"
$ git checkout -b v2 # optional but recommended
$ echo "module github.com/robteix/testmod/v2" > go.mod
$ git commit go.mod -m "Bump version to v2"
$ git tag v2.0.0
$ git push --tags origin v2 # or master if we don't have a branch

Upgrade major version


Even though we have released a new incompatible version of our library, existing programs have not broken down , because they continue to use version 1.0.1.
go get -uwill not download version 2.0.0.


But at some point, as a library user, I may want to upgrade to version 2.0.0, because, for example, I am one of those users who need support for several languages.


To upgrade, you need to change my program accordingly:


package main
import (
    "fmt""github.com/robteix/testmod/v2" 
)
funcmain() {
    g, err := testmod.Hi("Roberto", "pt")
    if err != nil {
        panic(err)
    }
    fmt.Println(g)
}

Now, when I launch go build, it "comes off" and will download for me version 2.0.0. Note that although the import path now ends with "v2", Go still refers to the module by its real name ("testmod").


As I said before, the major version is a different package in all respects. These two Go modules are unrelated. This means that we can have two incompatible versions in one binary:


package main
import (
    "fmt""github.com/robteix/testmod"
    testmodML "github.com/robteix/testmod/v2"
)
funcmain() {
    fmt.Println(testmod.Hi("Roberto"))
    g, err := testmodML.Hi("Roberto", "pt")
    if err != nil {
        panic(err)
    }
    fmt.Println(g)
}

And it eliminates the common problem with dependency management, when dependencies depend on different versions of the same library.



Let's go back to the previous version, which uses only testmod 2.0.0 - if we now check the content go.mod, we notice something:


modulemodrequire github.com/robteix/testmod v1.0.1require github.com/robteix/testmod/v2 v2.0.0

By default, Go does not remove dependencies from go.moduntil you ask for it. If you have dependencies that are no longer needed, and you want to clean them, you can use the new command tidy:


$ go mod tidy

Now we have only those dependencies that we really use.


Vending


Go modules ignore the directory by default vendor/. The idea is to gradually get rid of vendoring 1 . But if we still want to add "checked out" dependencies to our version control, we can do it:


$ go mod vendor

The team will create a directory vendor/in the root of our project, containing the source code of all dependencies.


However, go buildthe default is still ignoring the contents of this directory. If you want to collect dependencies from a directory vendor/, you need to explicitly ask about it.


$ go build -mod vendor

I assume that many developers who want to use vending will run go buildas usual on their machines and use -mod vendoron their CI.


I repeat, Go modules are moving away from the idea of ​​vendoring to using proxies for modules for those who do not want to directly depend on the upstream version control services.


There are ways to ensure that the gonetwork is not available (for example, using GOPROXY=off), but this is the topic of the next article.


Conclusion


The article may seem complicated to someone, but this is because I tried to explain a lot at once. The reality is that Go modules today are generally simple - we, as usual, import the package into our code, and the team does the rest for us go. Build dependencies are automatically loaded.


The modules also eliminate the need for $GOPATH, which was a stumbling block for new Go developers who had problems understanding why you need to put something in a particular directory.


Vending (unofficially) declared obsolete in favor of using a proxy.1
I can make a separate article about proxies for Go modules.


Notes:


1 I think this is too loud, and some may have the impression that the vendoring is being removed right now. This is not true. Vending still works, albeit slightly differently than before. Apparently, there is a desire to replace the vendor with something better, for example, a proxy (not a fact). For now, it's just a desire for a better solution. Vending will not go away until a good replacement is found (if there is one).

Tags:

Also popular now: