Go web development

The article is based on codelab from the Go site, but is not limited to it. As you read through the article, you can learn about Go data structures, dynamic arrays, using the http, template, regexp libraries to create a web application, template and filter input, respectively.

To understand the article, you need to be a little able to program, not to be scared by the words unix and the web. The basics of the language will be presented in the article.

Go installation

So, the first thing you need to work with Go is Linux, FreeBSD (OS X), although MinGW will work on Windows too.
Go needs to be installed, for this you need to do something like the following (instructions are given for systems with dpkg):

$ sudo apt-get install bison ed gawk gcc libc6-dev make # build tools
$ sudo apt-get install python-setuptools python-dev build-essential # build and install tools
$ sudo easy_install mercurial # if you don’t have it yet, it’s better not to install it from the repository (via apt)
$ hg clone -r release https://go.googlecode.com/hg/ go
$ cd go / src
$ ./all.bash

If all is well, you can add the following to ~ / .bashrc or ~ / .profile :

export GOROOT = $ HOME / go
export GOARCH = 386 # or amd64, depending on OS architecture
export GOOS = linux
export GOBIN = $ GOROOT / bin
PATH = $ PATH: $ GOBIN

When you login to the shell and run the compiler ( 8g for the i386 or 6g for amd64, will continue to 8g ) we get the help message:

gc: usage 8g [flags] file.go ...

This means that Go is installed and working, you can go directly to the application.

Start. Data structures

Create a directory for the application:

$ mkdir ~ / gowiki
$ cd ~ / gowiki

Create a wiki.go file with the following contents using a text editor ( binders for vim and emacs ) :

package main
import (
	"fmt"
	"io / ioutil"
	"os"
)

By name, it’s clear that our application will allow us to edit and save pages.
This code imports the fmt, ioutil, and os libraries from the Go standard library. Later we will add some other libraries.

Define several data structures. A wiki is a collection of linked pages with a body and a title. The corresponding data structure will have two fields:

type page struct {
	title string
	body [] byte
}

Data type [] byte  - this slice ( The slice ) type byte, analogue of dynamic array (for more information:  Effective the Go ) body of the article is stored in the [] byte , rather than string for ease of operation with standard libraries.

The data structure describes how data is stored in memory. But what if you need to save data for a long time? We implement the save method to save to disk:

func (p * page) save () os.Error {
	filename: = p.title + ".txt"
	return ioutil.WriteFile (filename, p.body, 0600)
}

The signature of this function reads: “This is the save method , applicable to the page pointer, without parameters, returning a value of type os.Error .”

This method will save the text to a file. For simplicity, we assume that the header is the file name. If this doesn't seem secure enough, you can import crypto / md5 and use the md5.New (filename) call .

The return value will be of type os.Error , corresponding to the return value of the WriteFile call (a standard library function for writing a slice to a file). This is done so that in the future it was possible to handle the error of saving to a file. If no problems, page.save ()will return us nil (null for pointers, interfaces, and some other types).

The octal constant 0600, the third parameter to the WriteFile call , indicates that the file is saved with read and write permissions only for the current user.

It would also be interesting to load the page:

func loadPage (title string) * page {
	filename: = title + ".txt"
	body, _: = ioutil.ReadFile (filename)
	return & page {title: title, body: body}
}

This function gets the file name from the header, reads the contents into a variable of type page, and returns a pointer to it.

Functions in Go can return multiple values. The io.ReadFile standard library function returns [] byte and os.Error . The function loadPage error is not handled: the underscore means "not store this value."

What happens if ReadFile returns an error? For example, there is no page with that title. This is a significant mistake; it cannot be ignored. Let our function also return two values: * page and os.Error .

func loadPage (title string) (* page, os.Error) {
	filename: = title + ".txt"
	body, err: = ioutil.ReadFile (filename)
	if err! = nil {
		return nil, err
	}
	return & page {title: title, body: body}, nil
}

Now you can check the value of the second parameter: if it is nil , then the page loaded successfully. Otherwise, it will be a value of type os.Error .

So, we have a data structure and methods of loading and unloading. It's time to check how this works:

func main () {
	p1: = & page {title: "TestPage", body: [] byte ("Test page.")}
	p1.save ()
	p2, _: = loadPage ("TestPage")
	fmt.Println (string (p2.body))
}

After compiling and executing this code, the TestPage.txt file will contain the value p1-> body . After that, this value is loaded into the p2 variable and displayed.

To build and run the program, you must do the following:

$ 8g wiki.go
$ 8l wiki.8
$ ./8.out
This is a sample page.

Http library

Go's simplest web server looks like this:

package main
import (
	"fmt"
	"http"
)
func handler (w http.ResponseWriter, r * http.Request) {
	fmt.Fprintf (w, "Hello% s!", r.URL.Path [1:])
}
func main () {
	http.HandleFunc ("/", handler)
	http.ListenAndServe (": 8080", nil)
}

The main function calls http.HandleFunc , which tells the http library that all kinds of requests ( "/" ) are handled by the handler function . By the

next call to http.ListenAndServe , we determine that we want to process requests on all interfaces on port 8080 ( ": 8080" ). The second parameter is not yet required. The program will work in this mode until forced termination.

Handler function of type http.HandlerFunc . It takes http.ResponseWriter and a pointer to http.Request as parameters .

A value of type http.ResponseWriterforms an http response ; writing data there (by calling Fprintf ) we return the contents of the page to the user.

The http.Request data structure is a user request. The string r.URL.Path  is the path. Suffix [1:] means "receive section Path (substring) from the first symbol to the end", i.e., to remove a leading slash.

By launching the browser and opening the URL http: // localhost: 8080 / habrahabr , we will see the desired page:

Hi habrahabr!

Using http to return pages

Import the http library :

import (
	"fmt"
	"http"
	"io / ioutil"
	"os"
)

Create a handler to display the article:

const lenPath = len ("/ view /")
func viewHandler (w http.ResponseWriter, r * http.Request) {
	title: = r.URL.Path [lenPath:]
	p, _: = loadPage (title)
	fmt.Fprintf (w, "
% s
% s
", p.title, p.body)
}

First, this function retrieves the header from r.URL.Path , the path components of the given URL. The global constant lenPath  is the length of the prefix "/ view /" in the path that indicates viewing the text of the article in our system. The substring [lenPath:] is highlighted , that is, the title of the article, the prefix is ​​excluded.

The function loads the data, supplementing it with simple html tags and writes to w , a parameter of type http.ResponseWriter . Reused

_ to ignore the return value of type os.Error. This is done for simplicity and generally not doing so well. Below you will find how to handle such errors correctly.

To call this handler, we write the main function that initializes http with the corresponding viewHandler to process requests along the path / view / .

func main () {
	http.HandleFunc ("/ view /", viewHandler)
	http.ListenAndServe (": 8080", nil)
}

Create a test page (in the file test.txt ), compile the code and try to display the page:

$ echo "Hello world"> test.txt
$ 8g wiki.go
$ 8l wiki.8
$ ./8.out

While our server is running, a page with the heading “test” containing the words “Hello world” will be available at http: // localhost: 8080 / view / test .

Change Pages

What kind of wiki is this without the ability to edit pages? Let's create two new handlers: editHandler to display the edit form and saveHandler to save the received data.

First, add them to main () :

func main () {
	http.HandleFunc ("/ view /", viewHandler)
	http.HandleFunc ("/ edit /", editHandler)
	http.HandleFunc ("/ save /", saveHandler)
	http.ListenAndServe (": 8080", nil)
}

Function editHandler loads the page (or creates an empty structure, if such a page is not present), and displays the form:

func editHandler (w http.ResponseWriter, r * http.Request) {
	title: = r.URL.Path [lenPath:]
	p, err: = loadPage (title)
	if err! = nil {
		p = & page {title: title}
	}
	fmt.Fprintf (w, "
Editing% s
"+
		"
"+
		"% s"+
		""+
		"
",
		p.title, p.title, p.body)
}

The function works well and correctly, but it looks ugly. The reason is in the html hardcode, but it is fixable.

Library template

The template library is part of the standard Go library. We can use templates to store html markup outside the code so that markup can be changed without recompilation.

First, import the template :

import (
	"http"
	"io / ioutil"
	"os"
	"template"
)

Create a form template in the edit.html file with the following contents:

Editing {title}
{body | html}

Change editHandler to use the template:

func editHandler (w http.ResponseWriter, r * http.Request) {
	title: = r.URL.Path [lenPath:]
	p, err: = loadPage (title)
	if err! = nil {
		p = & page {title: title}
	}
	t, _: = template.ParseFile ("edit.html", nil)
	t.Execute (p, w)
}

The template.ParseFile method will read the edit.html file and return a value of the type * template.Template .

The t.Execute method replaces all occurrences of {title} and {body} with the values ​​of p.title and p.body , and outputs the resulting html to a variable of type http.ResponseWriter.

Note that the template {body | html} was encountered in the template . It means that the parameter will be formatted for output in html, i.e. will be escaped and, for example, > will be replaced > . This will correctly display the data in the form.

Now call fmt.Sprintfnot in the program, you can remove fmt from import.

We will also create a template for displaying the page, view.html :

{title}
[ edit ]
{body}

Change viewHandler accordingly:

func viewHandler (w http.ResponseWriter, r * http.Request) {
	title: = r.URL.Path [lenPath:]
	p, _: = loadPage (title)
	t, _: = template.ParseFile ("view.html", nil)
	t.Execute (p, w)
}

Note that the code for calling templates is almost the same in either case. Let's get rid of duplication by taking this code into a separate function:

func viewHandler (w http.ResponseWriter, r * http.Request) {
	title: = r.URL.Path [lenPath:]
	p, _: = loadPage (title)
	renderTemplate (w, "view", p)
}
func editHandler (w http.ResponseWriter, r * http.Request) {
	title: = r.URL.Path [lenPath:]
	p, err: = loadPage (title)
	if err! = nil {
		p = & page {title: title}
	}
	renderTemplate (w, "edit", p)
}
func renderTemplate (w http.ResponseWriter, tmpl string, p * page) {
	t, _: = template.ParseFile (tmpl + ". html", nil)
	t.Execute (p, w)
}

Handlers are now shorter and simpler.

Handling Missing Pages

What happens when you click on the address / view / APageThatDoesntExist ? The program will fall. And all because we did not process the second value returned by loadPage . If the page does not exist, we will redirect the user to the page for creating a new article:

func viewHandler (w http.ResponseWriter, r * http.Request, title string) {
	p, err: = loadPage (title)
	if err! = nil {
		http.Redirect (w, r, "/ edit /" + title, http.StatusFound)
		return
	}
	renderTemplate (w, "view", p)
}

The http.Redirect function adds the HTTP status http.StatusFound (302) and the Location header to the HTTP response.

Saving Pages

The saveHandler function processes the data from the form.

func saveHandler (w http.ResponseWriter, r * http.Request) {
	title: = r.URL.Path [lenPath:]
	body: = r.FormValue ("body")
	p: = & page {title: title, body: [] byte (body)}
	p.save ()
	http.Redirect (w, r, "/ view /" + title, http.StatusFound)
}

A new page is created with the selected security and body. The save () method saves data to a file, the client is redirected to the / view / page .

The value returned by FormValue is of type string . To save it to the page structure, we convert it to [] byte by writing [] byte (body) .

Error processing

We ignore errors in our program in several places. This leads to the fact that the program crashes when an error occurs, so it is better to return an error message to the user, while the server will continue to work.

First, add error handling to renderTemplate :

func renderTemplate (w http.ResponseWriter, tmpl string, p * page) {
	t, err: = template.ParseFile (tmpl + ". html", nil)
	if err! = nil {
		http.Error (w, err.String (), http.StatusInternalServerError)
		return
	}
	err = t.Execute (p, w)
	if err! = nil {
		http.Error (w, err.String (), http.StatusInternalServerError)
	}
}

The http.Error function sends the selected HTTP status (in this case, “Internal Server Error”) and returns an error message.

Let's make a similar edit in saveHandler :

func saveHandler (w http.ResponseWriter, r * http.Request, title string) {
	body: = r.FormValue ("body")
	p: = & page {title: title, body: [] byte (body)}
	err: = p.save ()
	if err! = nil {
		http.Error (w, err.String (), http.StatusInternalServerError)
		return
	}
	http.Redirect (w, r, "/ view /" + title, http.StatusFound)
}

Any errors that occur in p.save () will be passed to the user.

Template Caching

Our code is not efficient enough: renderTemplate calls ParseFile every time the page is rendered. It is much better to call ParseFile once for each template at program startup, saving the received values ​​of * Template type in a structure for future use.

First, create a templates map in which we save the values ​​of * Template , the key in the map will be the name of the template:

var templates = make (map [string] * template.Template)

Next, we will create an initialization function, which we will call before main () . The template.MustParseFile function is a wrapper for ParseFile that does not return an error code; instead, it panics. Indeed, this behavior is acceptable for the program, because it is not known how to process the incorrect template.

func init () {for _, tmpl: = range [] string {"edit", "view"} {templates [tmpl] = template.MustParseFile (tmpl + ". html", nil)}}

The for loop is used with the range construct and processes the given patterns.

Next, change the renderTemplate function so that it calls the Execute method of the corresponding template:

func renderTemplate (w http.ResponseWriter, tmpl string, p * page) {
	err: = templates [tmpl] .Execute (p, w)
	if err! = nil {
		http.Error (w, err.String (), http.StatusInternalServerError)
	}
}

Validation

As already noted, there are serious security errors in our program. Instead of a name, you can pass an arbitrary path. Add a regular expression check.

Import the regexp library . Create a global variable in which we save our RV:

var titleValidator = regexp.MustCompile ("^ [a-zA-Z0-9] + $")

The regexp.MustCompile function compiles the regular expression and returns regexp.Regexp . MustCompile , like template.MustParseFile , differs from Compile in that it panics in the event of an error, while Compile returns an error code.

Now, let's build a function that extracts the title from the URL, and checks its PB titleValidator :

func getTitle (w http.ResponseWriter, r * http.Request) (title string, err os.Error) {
	title = r.URL.Path [lenPath:]
	if! titleValidator.MatchString (title) {
		http.NotFound (w, r)
		err = os.NewError ("Invalid Page Title")
	}
	return
}

If the title is correct, nil will be returned with it . Otherwise, “404 Not Found” will be displayed to the user, and an error will be returned to the handler.

Add a getTitle call to each of the handlers:

func viewHandler (w http.ResponseWriter, r * http.Request) {
	title, err: = getTitle (w, r)
	if err! = nil {
		return
	}
	p, err: = loadPage (title)
	if err! = nil {
		http.Redirect (w, r, "/ edit /" + title, http.StatusFound)
		return
	}
	renderTemplate (w, "view", p)
}
func editHandler (w http.ResponseWriter, r * http.Request) {
	title, err: = getTitle (w, r)
	if err! = nil {
		return
	}
	p, err: = loadPage (title)
	if err! = nil {
		p = & page {title: title}
	}
	renderTemplate (w, "edit", p)
}
func saveHandler (w http.ResponseWriter, r * http.Request) {
	title, err: = getTitle (w, r)
	if err! = nil {
		return
	}
	body: = r.FormValue ("body")
	p: = & page {title: title, body: [] byte (body)}
	err = p.save ()
	if err! = nil {
		http.Error (w, err.String (), http.StatusInternalServerError)
		return
	}
	http.Redirect (w, r, "/ view /" + title, http.StatusFound)
}

Functional types and closures

Checking errors and returns generates a fairly uniform code, it would be nice to write it only once. This is possible if, for example, you wrap functions that return errors in the corresponding call, the functional types will help us with this.

We rewrite the handlers by adding the title parameter :

func viewHandler (w http.ResponseWriter, r * http.Request, title string)
func editHandler (w http.ResponseWriter, r * http.Request, title string)
func saveHandler (w http.ResponseWriter, r * http.Request, title string)

We now define a wrapper function that takes the type of function defined above and returns http.HandlerFunc (to pass it to http.HandleFunc ):

func makeHandler (fn func (http.ResponseWriter, * http.Request, string)) http.HandlerFunc {
	return func (w http.ResponseWriter, r * http.Request) {
		// Here we will extract the page title from the Request,
		// and call the provided handler 'fn'
	}
}

The returned function is a closure, because uses values ​​defined outside of it (in this case, the variable fn , the handler). Let's move the

code from getTitle here :

func makeHandler (fn func (http.ResponseWriter, * http.Request, string)) http.HandlerFunc {
	return func (w http.ResponseWriter, r * http.Request) {
		title: = r.URL.Path [lenPath:]
		if! titleValidator.MatchString (title) {
			http.NotFound (w, r)
			return
		}
		fn (w, r, title)
	}
}

The closure returned by makeHandler is a function that takes parameters like http.ResponseWriter and http.Request (i.e., a function like http.HandlerFunc). This closure extracts the title from the URL and checks its PB titleValidator . If the header is incorrect, an error will be sent to ResponseWriter (call http.NotFound ). Otherwise, the corresponding fn handler will be called .

Add a wrapper call to the main () function :

func main () {
	http.HandleFunc ("/ view /", makeHandler (viewHandler))
	http.HandleFunc ("/ edit /", makeHandler (editHandler))
	http.HandleFunc ("/ save /", makeHandler (saveHandler))
	http.ListenAndServe (": 8080", nil)
}

Finally we remove the calls to getTitle from the handler functions, making them much simpler:

func viewHandler (w http.ResponseWriter, r * http.Request, title string) {
	p, err: = loadPage (title)
	if err! = nil {
		http.Redirect (w, r, "/ edit /" + title, http.StatusFound)
		return
	}
	renderTemplate (w, "view", p)
}
func editHandler (w http.ResponseWriter, r * http.Request, title string) {
	p, err: = loadPage (title)
	if err! = nil {
		p = & page {title: title}
	}
	renderTemplate (w, "edit", p)
}
func saveHandler (w http.ResponseWriter, r * http.Request, title string) {
	body: = r.FormValue ("body")
	p: = & page {title: title, body: [] byte (body)}
	err: = p.save ()
	if err! = nil {
		http.Error (w, err.String (), http.StatusInternalServerError)
		return
	}
	http.Redirect (w, r, "/ view /" + title, http.StatusFound)
}

That's what should happen as a result

Peresoberom code and run our application:

$ 8g wiki.go
$ 8l wiki.8
$ ./8.out

At http: // localhost: 8080 / view / ANewPage there will be a page with the form. You can save the page and go to it.

Note . textarea in the code had to be broken so as not to infuriate the habraparser.