November 2, 2015 at 02:00How to write Go code that is easy to port
- Transfer
(Translation of an article with tips on writing truly cross-platform code in Go)
Go is great for working with multiple platforms. My main development environment is on Windows, but I always work with Linux systems. Therefore, I naturally try to avoid things that can cause problems.
My attitude to cross-platform development is that if you consider yourself a serious developer, then your code should at least be built on other platforms , because even if not all functions can be used everywhere, some users will still want at least some of the functionality your library on other platforms.
I recently helped make the Windows version of a very nice backup program.since I wanted to learn alternatives to zpaq , a very good archiver with journaling and focus on compression. During porting, I noted several things that may be useful to others.
Let's start with the obvious. The syscall package is different for each platform, and although there are common points, you are almost guaranteed to run into trouble. Of course, there can be very good reasons for using it, but before using it, make sure that there are no other ways to do the same. If you use syscall / sys, immediately prepare that you will have to create separate files with the necessary assembly tags for porting, for example // + build darwin dragonfly freebsd linux netbsd openbsd solaris, and have a dummy implementation that will fill in the missing functionality during assembly for other platforms.
There is also a package github.com/golang/syswhich carries system calls into separate packages for each platform. This, however, does not solve the problem, so the same considerations apply here.
“No signal” by Πάνος Τσαλιγόπουλος
Signals are a very useful thing. There are tons of servers that use SIGHUP to reload the configuration, SIGUSR2 to restart the service, and so on. Please keep in mind that these signals are not available on other platforms, so do not make the main functionality depend on them . The web server without the examples above will work fine on Windows, even despite the lack of some functionality. Of course, it would be better to have a similar solution for Windows, but as long as the server compiles and works fine, I don’t think anyone will mind.
So if, say, you write a certain working service, you should not make a signal the only way to give a command to your service.
Remember that file systems are different.
The last point, by the way, is the most common mistake I met.
This is not a very obvious error, but since there is a simple example, it is easy to see that we are trying to delete the file before we close it.
The problem is that this works well on most systems, but on Windows it will fall. Here is a more correct example to do this:
As you can see, maintaining the order in which the file is closed is not a trivial task. If you choose an approach with two defers, remember that os.Remove must be defined before Close, as defer calls are executed in the reverse order.
Here is a more detailed article describing the differences between Windows and Linux file systems.
Using console commands to format output can be a good solution. This can help make the conclusion easier for visual perception, using colors or displaying progress without having to scroll through the text.
If you use ANSI codes, you should always use a translator library. Use it immediately and save yourself from a headache later. Here are some that I found in random order.
If you know any other good libraries, please write in the comments.
Symbolic links are a nice thing. It allows you to do cool things, like creating a new version of a file and just have a link that automatically points to the latest version of the file. But on Windows, symbolic links can only be created if the program has Administrator rights. So, although this is a nice little thing, try to ensure that the functionality of the program does not depend on it.
You should strive to avoid CGO as soon as you can, since setting up a working environment for building on Windows is quite difficult. If you use cgo, you refuse not only Windows, but also AppEngine users. If your code is a program, not a library, be prepared to upload binary files under Windows as well.
The same applies to external programs. Try to minimize their use for tasks that can be solved by libraries, and use external programs only for complex tasks.
Often, when working with some OS, you wonder how to write OS-specific code. It may be platform-specific code that is needed to satisfy some point. Take for example. next function; it does a lot of things, but one of the requirements is that on non-Windows platforms, the file must be created read-only. Let's look at two implementations:
This checks at runtime whether the program is running on Windows or not.
Compare this to:
The latest version is, as many would say, this should be done. I deliberately complicated the example to show that this might not always be the best solution. As for me, there may be times when the first option is preferable, especially if there is only one place for such code - and this is shorter, you do not need to look at several files to see what the code does.
I made a small tablet with the minuses and pluses of each approach:
In general, I would recommend checking at compile time, with different files, except, perhaps, when you write a test. But in certain cases, you may prefer checking at runtime.
As a conclusion, configure cross-platform tests. This is one of the useful things that I learned from restic , from which cross compilation has already been configured. When Go 1.5 is released, cross-platform compilation will be even easier, as it will require even less body movements to configure.
At the same time, and for older versions of Go, you can look at gox , which helps automate cross-compilation. If you need even more advanced features, check out goxc .
Happy coding!
From the author:
Go is great for working with multiple platforms. My main development environment is on Windows, but I always work with Linux systems. Therefore, I naturally try to avoid things that can cause problems.
My attitude to cross-platform development is that if you consider yourself a serious developer, then your code should at least be built on other platforms , because even if not all functions can be used everywhere, some users will still want at least some of the functionality your library on other platforms.
I recently helped make the Windows version of a very nice backup program.since I wanted to learn alternatives to zpaq , a very good archiver with journaling and focus on compression. During porting, I noted several things that may be useful to others.
Minimize the use of syscall (or sys)
Let's start with the obvious. The syscall package is different for each platform, and although there are common points, you are almost guaranteed to run into trouble. Of course, there can be very good reasons for using it, but before using it, make sure that there are no other ways to do the same. If you use syscall / sys, immediately prepare that you will have to create separate files with the necessary assembly tags for porting, for example // + build darwin dragonfly freebsd linux netbsd openbsd solaris, and have a dummy implementation that will fill in the missing functionality during assembly for other platforms.
There is also a package github.com/golang/syswhich carries system calls into separate packages for each platform. This, however, does not solve the problem, so the same considerations apply here.
Try not to depend on signals
“No signal” by Πάνος Τσαλιγόπουλος
Signals are a very useful thing. There are tons of servers that use SIGHUP to reload the configuration, SIGUSR2 to restart the service, and so on. Please keep in mind that these signals are not available on other platforms, so do not make the main functionality depend on them . The web server without the examples above will work fine on Windows, even despite the lack of some functionality. Of course, it would be better to have a similar solution for Windows, but as long as the server compiles and works fine, I don’t think anyone will mind.
So if, say, you write a certain working service, you should not make a signal the only way to give a command to your service.
File System Differences
Remember that file systems are different.
- Most operating systems have case-sensitive file systems, but not Windows. My advice: always use lowercase
- Remember os.PathSeparator . Always use it, but this is not the only possible delimiter. Windows can use both "/" and "\", so the paths received from the user can contain both.
- Always use the filepath package . It may be a little more code, but you will save yourself and others from a headache.
- os.Remove and os.RemoveAll cannot delete Read-only files on Windows. This is a bug and should have been fixed a long time ago. Unfortunately: politics.
- os.Link, os.Symlink, os.Chown, os.Lchown, os.Fchown return errors in Windows. These errors, however, are exported only to Windows .
- os / user.Current will not work if the binary is cross-compiled. Look here . Thanks @njcw
- And always close the files after changing / deleting them.
The last point, by the way, is the most common mistake I met.
func example() (err error) {
var f *os.File
f, err = os.Create("myfile.txt")
if err != nil {
return
}
defer f.Close()
err = f.write([]byte{"somedata"})
if err != nil {
return
}
// Do more work...
err = os.Remove("myfile.txt")
}This is not a very obvious error, but since there is a simple example, it is easy to see that we are trying to delete the file before we close it.
The problem is that this works well on most systems, but on Windows it will fall. Here is a more correct example to do this:
func example() (err error) {
var f *os.File
f, err = os.Create("myfile.txt")
if err != nil {
return
}
defer func() {
f.Close()
err2 := os.Remove("myfile.txt")
if err == nil && err2 != nil {
err = err2
}
}()
err = f.write([]byte{"somedata"})
// Do more work
}As you can see, maintaining the order in which the file is closed is not a trivial task. If you choose an approach with two defers, remember that os.Remove must be defined before Close, as defer calls are executed in the reverse order.
Here is a more detailed article describing the differences between Windows and Linux file systems.
Use translator if using ANSI
Using console commands to format output can be a good solution. This can help make the conclusion easier for visual perception, using colors or displaying progress without having to scroll through the text.
If you use ANSI codes, you should always use a translator library. Use it immediately and save yourself from a headache later. Here are some that I found in random order.
- Azure / go-ansiterm - used in Docker
- mgutz / ansi - an abstraction of ANSI codes
- fatih / color - seems to only deal with colors
If you know any other good libraries, please write in the comments.
Avoid dependency on symbolic links
Symbolic links are a nice thing. It allows you to do cool things, like creating a new version of a file and just have a link that automatically points to the latest version of the file. But on Windows, symbolic links can only be created if the program has Administrator rights. So, although this is a nice little thing, try to ensure that the functionality of the program does not depend on it.
Avoid CGO and external programs if possible.
You should strive to avoid CGO as soon as you can, since setting up a working environment for building on Windows is quite difficult. If you use cgo, you refuse not only Windows, but also AppEngine users. If your code is a program, not a library, be prepared to upload binary files under Windows as well.
The same applies to external programs. Try to minimize their use for tasks that can be solved by libraries, and use external programs only for complex tasks.
At compile time or at runtime?
Often, when working with some OS, you wonder how to write OS-specific code. It may be platform-specific code that is needed to satisfy some point. Take for example. next function; it does a lot of things, but one of the requirements is that on non-Windows platforms, the file must be created read-only. Let's look at two implementations:
func example() {
filename := "myfile.txt"
fi, _ := os.Stat(f)
// set file to readonly, except on Windows
if runtime.GOOS != "windows" {
os.Chmod(f, fi.Mode()&os.FileMode(^uint32(0222)))
}
}This checks at runtime whether the program is running on Windows or not.
Compare this to:
func example() {
filename := "myfile.txt"
fi, _ := os.Stat(f)
setNewFileMode(f, fi)
}
// example_unix.go
//+build !windows
// set file to readonly
func setNewFileMode(f string, fi os.FileInfo) error {
return os.Chmod(f, fi.Mode()&os.FileMode(^uint32(0222)))
}
// example_windows.go:
// we don't set file to readonly on Windows
func setNewFileMode(f string, fi os.FileInfo) error {
return nil
}The latest version is, as many would say, this should be done. I deliberately complicated the example to show that this might not always be the best solution. As for me, there may be times when the first option is preferable, especially if there is only one place for such code - and this is shorter, you do not need to look at several files to see what the code does.
I made a small tablet with the minuses and pluses of each approach:
| Pros "at compile time" | Pros "at runtime" |
| Minimal or missing overhead | You can keep all the code in one place |
| Code for each platform in a separate file | Some errors can be detected without cross-compiling. |
| Can use imports that do not compile on all platforms |
| Cons "at compile time" | Cons "at runtime" |
| Code duplication may be required. | There is no easy way to see where platform-specific code is |
| May lead to many small files and whether to one large file with a bunch of scattered functionality | Slight overhead to check |
| To view the code, you need to open several files | You cannot use structures / functions that are not platform independent |
| You need to use cross-compilation to make sure the code is going to |
In general, I would recommend checking at compile time, with different files, except, perhaps, when you write a test. But in certain cases, you may prefer checking at runtime.
CI configuration for cross-platform tests
As a conclusion, configure cross-platform tests. This is one of the useful things that I learned from restic , from which cross compilation has already been configured. When Go 1.5 is released, cross-platform compilation will be even easier, as it will require even less body movements to configure.
At the same time, and for older versions of Go, you can look at gox , which helps automate cross-compilation. If you need even more advanced features, check out goxc .
Happy coding!
From the author:
- For the translation, you can thank the lair hub user - the more his inadequate comments about Go appear in the hub, the more translations and articles about Go will be.
- An article about the basics of cross-compilation in Go - habrahabr.ru/post/249449