Go (golang)

(Created 2016. Revised 2018, 2019.)

Go is an open source programming language developed by Rob Griesemer, Rob Pike, and Ken Thompson, and released by Google. Rob Pike and Ken Thompson are Bell Labs alumni, who helped invent Unix, Plan 9, UTF-8, etc.

The Go gopher mascot resembles Glenda, the Plan 9 mascot. Both were created by Renee French.

“Go” is commonly written as “golang” to make it searchable.

Go is a compiled language. Two compilers are available: Google’s “gc” compiler and GNU’s gccgo. Almost all developers use the Google compiler. Go compiles to stand-alone, self-contained binaries. Go compilation is fast.

Go is strongly and statically typed. Variables are of a particular type (int, string, bool, etc.), which is fixed when they’re declared. A rigid type system helps the compiler detect problems and make optimizations.

Go’s syntax is largely C-like. Indexes start at zero, == is the equality test operator, curly braces delimit a scope, execution starts in main, and so forth. Unlike C, if conditions don’t need parentheses, and statements are not terminated by semi-colons.

Go is garbage collected. The garbage collector is fast. Garbage collection pauses are insignificant for almost all use cases.

One of the key features of Go is a commitment to simplicity.

Go ships official binaries for the FreeBSD, Linux, macOS, and Windows on 32-bit (386) and 64-bit x86. Ports exist for a number of additional platforms and architectures (OpenBSD, arm64, etc.).

Learn Go with:

A Tour of Go
Alan Donovan and Brian Kernighan’s book The Go Programming Lanuage

Installation/Updates, Environment, Paths, and the Go Workspace

Installation and updates of Go work essentially the same way: download Go and unpack it.

When updating, clear out the old version with:

# rm -r /usr/local/go/

Download a binary release from:

https://golang.org/dl/

Un-archive the download:

# tar -C /usr/local -xzf go1.7.5.linux-amd64.tar.gz

When first installing Go, set a couple of environment variables. Add /usr/local/go/bin to your path (and shell rc file), and set $GOPATH.

$ export PATH=$PATH:/usr/local/go/bin
$ export GOPATH=$HOME/go

The Go environment is scoped to a workspace — a directory with three sub-directories at its root: src, pkg, and bin. The GOPATH environment variable defines the location of the workspace. Go developers typically use only a single workspace, with each project in a src subdirectory, like $HOME/go/src/paulgorman.org/helloworld/.

NOTE: As of late 2018, $GOPATH is fading away as part of Go’s new module system.

Hello, world

$  mkdir -p $GOPATH/src/github.com/joeuser/hello
$  vim $GOPATH/src/github.com/joeuser/hello/hello.go

	package main

	import (
		"fmt"
		"math"
	)

	func add(x int, y int) int {
		return x + y
	}

	func main() {
		fmt.Printf("Hello, world!\n")
		fmt.Println(math.Pi)
		fmt.Println(add(3, 4))
	}

$  go install github.com/joeuser/hello
$  $GOPATH/bin/hello
Hello, world!

Example Code

The Go package sources include excellent example code:

https://golang.org/src/

REPL (Go Playground)

Go currently doesn’t provide a local REPL, but the web-based Go Playground helps:

https://play.golang.org/

The Go Playground is also a good (and common) way to share code, especially when asking questions.

Documentation

https://golang.org/doc/

In addition to the official website, Go offers two local documentation tools:

the command-line go doc tool
the local web doc server godoc

Start a local documentation web server at http://localhost:808:

$  godoc -http=:8080

Or, access docs or Go source code from the command line with go doc:

$  go doc fmt | grep 'uint'
	uint, uint8 etc.:        %d, %#x if printed with %#v
$  go doc fmt Println
$  go doc -src fmt Println

Show the docs for the package in the current directory:

$  go doc .

[Go 1.11, deprecated godoc as a command line tool. This will eventually simplify things. As of 2018, go doc is not yet a complete replacement for godoc; it lacks the -src flag, for example.]

Comments (and `go doc` Commentary)

// Double front-slashes denote a comment that runs to the end of the line.
/* A slash
   and a splat
   starts a multi-line comment that ends with
   a splat and a slash.
*/

Go’s go doc tool generates documentation from comments:

go doc processes Go source files to extract documentation about the contents of the package. Comments that appear before top-level declarations, with no intervening newlines, are extracted along with the declaration to serve as explanatory text for the item.

Every package should have a “package comment” preceding the package clause, like // Package foo does bar. For multi-file packages, the package comment only needs to be present in one file, and any one will do. The package comment should introduce the package and provide information relevant to the package as a whole. It will appear first on the godoc page and should set up the detailed documentation that follows.

Inside a package, any comment immediately preceding a top-level declaration serves as a “doc comment” for that declaration. Every exported (capitalized) name in a program should have a doc comment.

The first sentence of a doc comment should start with the name of the declared item and summarize its function:

// Compile parses a regular expression and returns, if successful, a Regexp that can be used to match against text.
func Compile(str string) (*Regexp, error) {
	// …
}

Code Style (`go fmt`)

In general, Go is an opinionated language, and that applies to code formatting. Go supplies the go fmt utility to automatically apply the official code style. No more arguing about tabs versus spaces.

$ go fmt mything.go

Basic Types and Variables

Go’s integer types are int8, int16, int32, and int64, along with the unsigned types uint8, uint16, uint32, and uint64. Three additional types — uint, int, and uintptr — are architecture-dependent in size. The int gets used most.

The type byte is an alias to uint8. The rune type is an alias to int32.

Go offers two types for floating-point numbers: float32 and float64. The complex number types are complex64 and complex128.

Go’s string type may be treated as an array of bytes (one byte per character, except in languages with multi-byte characters). Specify string literals with “double quotes” or backticks. Go interprets escapes like “\n” for newline inside double-quotes but not in backticks.

Go has a boolean type for true and false (effectively a 1-bit integer).

That’s it for Go’s simple types. See the sections below for complex types, like slices, maps, and structs.

Variable names must begin with a letter, and may include letters, numbers, or underscores. By convention, variable names should be camelCase/BumpyCase. Lower-case names are local to a package, while upper-case names are visible outside the package (see the “Packages” section below).

Declare variables then assign values, or declare and assign in one go:

var x int
x = 100
var y int = 10

Unlike in C, type comes after the variable name in declarations.

Go infers type for declarations using the := operator to assign a value:

x := 100

…but a variable can’t be declared twice. Subsequent assignments to x must use the = operator:

x := 100
x = 2

With multiple variables, as long as at least one is newly declared, := works:

x, y, z := 9, "UFO", 42

Declared variables must be used, or compilation fails.

Declared but unassigned variables get an appropriate default value — zero for numeric types, false for booleans, “” (empty) for strings.

Go makes it easy to turn strings into byte arrays and the reverse:

s := "Keep off the moors!"
bytearray := []byte(s)
s2 := string(bytearray)

A string can be treated like a read-only slice of bytes.

Note that bytes in a string are not necessarily valid ASCII, or even valid UTF-8. Go strings often contain UTF-8 bytes, but not always; strings can contain arbitrary bytes. What Unicode calls “code points”, Go calls “runes”.

var r rune = 'a'

Go uses += operator for string concatenation. However, += creates a new string. For large or repeated uses, consider Join from the strings package instead of +=.

s += " bar"
strings.Join(os.Args[1:], " ")

See section “3.5.4. Strings and Byte Slices” of The Go Programming Language:

Because strings are immutable, building up strings incrementally can involve a lot of allocation and copying. In such cases, it’s more efficient to use the bytes.Buffer type[…]

Go contains raw string literals in backticks rather than double-quotes. Go ignores escape sequences in raw string literals. Raw string literals are useful for defining regular expressions, for example, or embedding templates in code, like a Perl heredoc.

x := `This is
a raw string
literal.`

Create constants with a fixed value using the const keyword:

const pi float32 = 3.14159

Like C, Go lexically scopes variables to blocks (i.e., they exist within the bounds of curly braces).

Flow Control Statements: for, if, else, and switch

for is Go’s only loop.

for i := 0; i < 10; i++ {
	sum += i
}

“Initialization” (i := 0) runs before the loop starts. Initialization must be a simple statement, like a variable declaration or a function call.
The “condition” (i < 10) is a boolean expression evaluated at the start of each loop iteration.
The “post” (i++) statement runs after each iteration of the loop body.

Don’t use parentheses around the “init; condition; post”.

Omitting the optional “init” and “post” statements acts like C’s while loop:

sum := 1
for sum < 1000 {
	sum += sum
}

Loop forever by omitting even the “condition”:

for {
	foo()
}

Iterate over a slice:

my_slice := []string{"a", "b", "c", "d"}
for index, value := range my_slice {
	fmt.Println(index, " : ", value)
}

Note that range gives a copy of the value, so changes made to the copy don’t affect the original value outside the loop.

my_slice := []string{"a", "b", "c", "d"}
for index, value := range my_slice {
	value = "x" // "my_slice" is unchanged.
	my_slice[index] = "x" // "my_slice" changed.
}

The continue keyword jumps to the next iteration of the for loop. The break keyword terminates the loop.

Like for, omit the parentheses for if. if can start with multiple short statements:

if x := blerg(); x < 0 {
	return x + 1
} else if y == 99 {
	return y
} else {
	return x - 1
}

Values from the start of an if (like “x” above) are scoped to the if block (and any else block).

Each case in a switch breaks implicitly (unless it ends with fallthrough). Like if, ‘switch’ can have multiple short preliminary statements.

fmt.Print("Go runs on ")
switch os := runtime.GOOS; os {
case "darwin":
	fmt.Println("macOS.")
case "linux":
	fmt.Println("Linux")
default:
	fmt.Printf("%s.", os)
}

Without a condition, switch acts like switch true. It can be used for long if-then-else chains.

Functions

Functions can return multiple values. This function returns an int and a boolean:

func vaporize(name string) (int, bool) {
	// …
}

Discard uninteresting return values by assigning them to an underscore, like:

_, b = vaporize("bar")

The underscore is more than a convention. No assignment actually occurs with _, and it doesn’t care about type.

Functions are first-class types. They can be passed as arguments, or returned as values, or used as a field type in a struct. We can define new function types:

type Add func(a int, b int) int

Functions form call stacks. When we call a function, Go pushes it on top of the call stack. When the function returns, Go pops that top function off the stack

Go has “function literals” (a.k.a. anonymous functions):

f := func(x, y int) int { return x + y }

Function literals are closures. They share variables defined in their surrounding function. Those shared variables survive as long as they are accessible.

Defer

Go’s defer keyword delays execution of a function until return of the enclosing function. Defer mainly helps with “clean-up” tasks.

package main

import (
	"fmt"
	"os"
)

func main() {
	file, err := os.Open("myfile")
	if err != nil {
		fmt.Println(err)
		return
	}
	defer file.Close()
	// … Read the file, etc.
}

Arguments to the deferred function are evaluated immediately (not when the surrounding function returns). Deferred function calls get pushed onto a stack; when a function returns, the deferred functions get called in last-in-first-out order.

Variadic Functions

A “variadic” function may be called with a varying number of arguments. fmt.Printf is an example of a variadic function.

When declaring a variadic function, precede the final parameter type with an ellipsis (“…”).

func sum(vals ...int) int {
	total := 0
	for _, val := range vals {
		total += val
	}
	return total
}

To invoke a variadic function when the arguments are already in a slice, append an ellipsis to the argument. These two invocations of sum() are effectively the same:

sum(1, 2, 3, 4)
values := []int{1, 2, 3, 4}
sum(values...)

See The Go Programming Language section 5.7.

Errors (and `recover` and `panic`)

Go functions often return a value of type error as their final return value. If all goes well, the value is nil.

See the “Error Handling” section below.

Stucts and Pointers

Go passes by value. If we pass a variable to a function, a copy of the value is created that lives for the scope of the function. Any changes to the copy vanish when the function exits.

By passing a pointer into the function instead, changes to the value survive beyond the scope of the function. Furthermore, passing a pointer to a memory address is more efficient than copying values.† We often want to pass a pointer instead of copying values.

† EDIT/ASIDE: Is passing pointers actually more efficient, or does it really lead to cache misses and expensive heap allocations? Probably it’s generally best to at least pass scalar data (single values, rather than structs) by value. And since slices, for example, are already basically pointers, passing them by reference simply creates an unnecessary additional layer of indirection. https://www.reddit.com/r/golang/comments/a410gl/go_is_passbyvalue_does_that_mean_each_function/ebb12dw/

Like in C, & is the “address-of” operator, and * is the “value-at” operator. More properly, the & operator returns a pointer to its operand. &T returns a pointer to T. The * operator returns the value of its operand. *T returns the value of T.

Create pointers by declaring a variable of a pointer type or by using the new function. Both methods allocate memory for the type and return a pointer to it, but new also initializes it to a zero value.

x := new(int) // `new` allocates memory and initializes it to a zero value.
var y *int // `y` points to int-size memory, but the value there remains `nil`.
fmt.Println(x) // 0xc420016100
fmt.Println(*x) // 0
fmt.Println(&y) // 0xc420084018
fmt.Println(y) // Nil

Like C, Go has structures. Access struct fields using the dot (.) operator.

type Car struct {
	Model string
	Wheels int
	Speed int
}

mycar := Car{
	Model: "Beetle",
	Wheels: 4, // Note *required* trailing comma!
}
secondcar := Car{}
secondcar.Wheels = 3

thirdcar := &Car{"Hexamobile", 6, 0} // Pointer to new Car.

Go structures can receive methods, making them a bit like objects.

func (c *Car) Accelerate() { // *Car is the receiver of the Accelerate() method.
	c.Speed += 10
}

thirdcar.Accelerate()

(Any Go type can receive methods.)

Go structs support embedded types. These are awkward to use in some ways, but they allow us to call methods of the embedded type on the parent struct.

type Dunebuggy struct {
	Car
	Grooviness int
}

// …

b := Dunebuggy{
	Car: Car{Wheels: 4,},
	Grooviness: 19,
}
fmt.Println(b.Wheels)
b.Accelerate()
fmt.Println(b.Speed)

Anonymous structs:

car := struct {
	make   string
	model  string
	color  string
	weight float64
	speed  float64
}{
	"Ford",
	"Destroyer",
	"orange",
	3628.7,
	227.4,
}
fmt.Println(car.speed)

Use struct tags to appease JSON consumers that want lowercase key names:

type Animal struct {
	Species string `json:"species"`
	Color   string `json:"color"`
}

Composition

Instead of hierarchical inheritance, Go supports composition. Compose hybrid types by including one struct type in another. (Some languages call this a “mix-in” or “trait”.)

type Vehicle struct {
	Make string
	Model string
}

type Bicycle struct {
	*Vehicle
	Wheels int
	Riders int
}

func (b *Bicycle) String() string {
	return fmt.Sprintf(
		"Make: %s, Model: %s, Wheels: %d, Riders: %d",
		b.Vehicle.Make, b.Model,    // Go exposes both "b.Vehicle.Model" and the shorter "b.Model".
		b.Wheels, b.Riders,
	)
}

func main() {
	mybike := &Bicycle{
		Vehicle: &Vehicle{"Raleigh", "Tourist"},
		Wheels: 2,
		Riders: 1,
	}
	fmt.Println(mybike)
}

Overloading is the ability to create multiple methods with the same name differentiated either by type of argument or number of arguments. Go does not support overloading. However, we can overwrite the methods of a mixed-in type (and if necessary still call an overwritten method like thing.Mixin.foo()).

Factories not Constructors

Go doesn’t have constructors for objects, but the factory pattern helps create new structs.

type Car struct {
	Model string
	Wheels int
	Speed int
}

func NewCar(model string, wheels int, speed int) *Car {
	return &Car{    // We return a pointer to the new struct.
		Model: model,
		Wheels: wheels,
		Speed: speed,
	}
}

func main() {
	monsterTruck := NewCar("Crusher", 4, 1000000)
	fmt.Println(monsterTruck.Speed)
}

Default Function Arguments

https://talks.golang.org/2012/splash.article

One feature missing from Go is that it does not support default function arguments. This was a deliberate simplification. Experience tells us that defaulted arguments make it too easy to patch over API design flaws by adding more arguments, resulting in too many arguments with interactions that are difficult to disentangle or even understand. The lack of default arguments requires more functions or methods to be defined, as one function cannot hold the entire interface, but that leads to a clearer API that is easier to understand. Those functions all need separate names, too, which makes it clear which combinations exist, as well as encouraging more thought about naming, a critical aspect of clarity and readability.

Arrays, Slices, and Maps

The size of an array never changes after creation.

var counts [10]int    // Create an array
counts[0] = 102    // Assign a value to an array.

weights := [3]int{3492, 487, 48762}    // Create and initialize an array

for index, value := range weights {    // Iterate over an array.
	// …
}

How arrays in Go differ from C array:

Go arrays are values; assigning one array to another copies all the elements.
Passing an array to a function passes a copy, not a pointer to the array. To pass a pointer, be explicit (dothing(&myarray)).
Size is part of an array’s type (e.g. [4]int is a distinct type from [10]int).

In most cases, use a slice instead of an array.

Assigning one slice to another create a reference rather than copying values.
Passing a slice to a function passes a reference rather than copying the values.

Slices are an abstraction on top of arrays, providing growable lists. An array has a length (findable with the len function). A slice has both a length and a capacity (findable with the cap function). The slice length is the current number of elements allocated. The slice capacity is the size of the array underlying the slice.

weights := []int{234, 3458435, 223, 23432}    // Create and populate a slice.
weights := make([]int, 10)    // Create a slice with length 10 and capacity 10 (values initialized to "0").
weights := make([]int, 0, 10)    // Create a slice with length 0 and capacity 10.

If we force Go to grow the capacity of a slice with append(), it automatically copies all the values into a larger array. Go uses a two-times algorithm — each time it grows the slice, the new array doubles the size of the old one.

a := make([]int, 5)
fmt.Println(len(a), cap(a))    // 5 5

a = append(a, 1)
fmt.Println(len(a), cap(a))    // 6 10

for i := 0; i < 5; i++ {
	a = append(a, 1)
}
fmt.Println(len(a), cap(a))    // 11 20

for i := 0; i < 10; i++ {
	a = append(a, 1)
}
fmt.Println(len(a), cap(a))    // 21 40

Go also accepts slice notation, (although not negative notation, like x[:-2]).

x := []int{2, 4, 8, 12, 16, 20, 24}
fmt.Println(x[2:5])    // 8 12 16
fmt.Println(x[:3])    // 2 4 8
fmt.Println(x[5:])    // 20 24
y := make([]int, 3)
copy(y, x[3:6])    // Copy part of one slice to another.
fmt.Println(y)    // 12 16 20

The copy function creates entries in a destination slice to match the source slice, overwriting any existing destination entries:

s1 := []int{0, 1, 2, 3}
s2 := make([]int, 4)
copy(s2, s1) // Both slices now hold [0 1 2 3].

Maps hold keys and values (like dictionaries in Python). Initialize an unpopulated map with make, like:

m = make(map[string]int)

Maps grow dynamically, but make accepts an optional second argument for initial length. The delete function removes items from the map.

ages := make(map[string]int)
ages["Ben"] = 26
ages["Richard"] = 31
fmt.Println(ages["Ben"])
fmt.Println(len(ages))    // 2

delete(ages, "Ben")
fmt.Println(len(ages))    // 1

codenames := map[string]string {    // Another way to declare and initialize maps.
	"Ben": "Brown Boxer",
	"Richard": "Steam Shovel",
	"Phil": "Panther",
	"Anne": "Seatle",
}

for key, value := range codenames {
	fmt.Println(key, "\t:\t", value)
}

type Monstertruck struct {
	Name string
	Tirepressure map[string]int
}

mytruck := &Monstertruck {
	Name: "King Krusher",
	Tirepressure: map[string]int {
		"Front-left": 100,
		"Front-right": 99,
		"Rear-left": 100,
		"Rear-right": 97,
	},
}

fmt.Println(mytruck.Tirepressure["Rear-right"])

Writing to a nil map will cause a runtime panic; don’t do that.

A map retrieval yields a zero value when the key is not present. Or, check for a second “ok” value to test the result of a map lookup:

if capitals["Michigan"] != "" {
	fmt.Println(capitals["Michigan"])
}

if city, ok := capitals["Michigan"]; ok {
	fmt.Println(city, ok) // -> "Lansing" true
}

When iterating over a map with a range loop, the order is not guaranteed to be the same from one run to the next.

Maps are not safe for concurrent use. One common way to protect maps is with sync.RWMutex. This, for example, declares an anonymous struct containing a map and an embedded sync.RWMutex:

var counter = struct{
	sync.RWMutex
	m map[string]int
}{m: make(map[string]int)}

To read from the counter, take the read lock:

counter.RLock()
n := counter.m["some_key"]
counter.RUnlock()
fmt.Println("some_key:", n)

To write to the counter, take the write lock:

counter.Lock()
counter.m["some_key"]++
counter.Unlock()

New and Make

Go had two allocation primitives: new and make. They work on different types, and do slightly different things.

new is a built-in function. It allocates memory. However, rather than initialize that memory, new only zeroes it. So, new(T) allocates and zeroes enough memory to hold a value of type T. new returns a pointer to this newly-allocated, zero-value T.

Where possible, design data structures usable with the zero-value of each type, without further initialization. For example, the zero-value of bytes.Buffer give an empty Buffer ready for use.

Unlike new, the built-in function make works only on slices, maps, and channels. Also unlike new, make(T, args) return an initialized (non-zeroed) value of type T. Go uses make for these types because slices, maps, and channels always require initialization before use.

var p *[]int = new([]int)       // This allocates a slice structure. Since `*p == nil`, this is rarely useful.
var v  []int = make([]int, 100) // The slice `v` now refers to a new array of 100 `int`s.

// Unnecessarily complex:
var p *[]int = new([]int)
*p = make([]int, 100, 100)

// Idiomatic:
v := make([]int, 100)

https://golang.org/doc/effective_go.html#data

Error Handling

Note: error handling changed slightly in Go 1.13.

Go handles errors with return values rather than exceptions. Create custom errors by following the error interface:

type error interface {
	Error() string
}

… or import the errors package:

import "errors"

func pack(results int) error {
	if results < 1 {
		return errors.New("Too few results to pack.")
	}
	// …
	return nil    // We test the return value like `if err != nil { … }`.
}

Section 5.4 of The Go Programming Language is required reading. It describes five strategies of error writing.

The first (and most common) strategy is to propagate the error, so a failure in a subroutine bubbles up to the calling routine:

resp, err := http.Get(url)
if err != nil {
	return nil, err
}

We may need to add contextual information to the error before propagating it upward:

doc, err := html.Parse(resp.Body)
resp.Body.Close()
if err != nil {
	return nil, fmt.Errorf("parsing %s as HTML: %w", url, err)
}

(The %w verb in Errorf came in Go 1.13, and allows use of errors.Unwrap, errors.Is, and errors.As.)

fmt.Errorf acts like fmt.Sprintf, but returns an error type value. Prefixing additional error information at each level of method call produces a useful chain of information. Because error messages are often chained together, when writing error messages:

Don’t capitalize like a sentence.
Don’t add line breaks.

Function f(x) should return errors report the attempted operation f and the value of x in context.

The second error strategy is to retry some number of times, possibly waiting between each attempt. This makes sense for transient problems or delays that may resolve themselves.

The third error strategy is to log the error and gracefully end the program. Try to do this only from the main package.

log.Fatalf("Bad juju: %v", err)

The forth error strategy is to log the error and continue, possibly with reduced functionality.

In the fifth case, we can choose to safely ignore some errors.

Some very simple functions, where only one thing might possibly go amiss, return a boolean value rather than an error:

value, ok := cache.Lookup(key)
if !ok {
	// … cache[key] does not exist.
}

`panic` and `recover`

The recover function helps to handle or clean up after runtime panics. It returns the panic message.

func main() {
	defer func() {
		p := recover()
		fmt.Println(p)
	}()
	panic("Holy moly!")
}

Packages

NOTE: As of late 2018, Go’s new module system is starting to significantly change package handling, dependencies, and versioning.

Packages group together related code, and hide private details from other packages. Functions, variables, and types with uppercase names are visible outside the package. Functions, variables, and types with lowercase names are only visible within the package.

Package names follow the directory structure relative to $GOPATH. Import a package at $GOPATH/src/example.com/foo like import "example.com/foo". Import a package at $GOPATH/src/example.com/foo/bar like import "example.com/foo/bar". (Though files in package “bar” will use the header “package bar” not “example.com/foo/bar” or “foo/bar”.)

Go allows the use of an alias for packages with long names or conflicting names.

We import some packages only for the side-effects — e.g., import triggers the package’s init function that fulfills our needs. Use an underscore (_) alias when importing such packages.

Group multiple imports like:

import (
	"fmt"
	"math"
	"os"

	m "example.com/math"
	_ "github.com/mattn/go-sqlite3"
)

go get pulls remote dependencies, and stores them in the Go workspace:

$ go get example.com/paulgorman/example

It can update one package or all previously downloaded packages:

$ go get -u example.com/paulgorman/example
$ go get -u

got get has limitations. It relies on git, and it always points to the master/head/trunk/default rather than a particular version. A number of more sophisticated package managers are under development, but none has emerged as a clear winner.

https://talks.golang.org/2012/splash.article

One may always tell whether a name is local to package from its syntax: Name vs. pkg.Name. It’s important to recognize that package paths are unique, but there is no such requirement for package names. The package name need not be unique and can be overridden in each importing source file by providing a local identifier in the import clause Every company might have its own log package but there is no need to make the package name unique. Quite the opposite: Go style suggests keeping package names short and clear and obvious in preference to worrying about collisions.

Interfaces

Interfaces do a lot of work in Go. An interfaces specifies the type of an object by its behavior — if something can do this, it can be used here.

Interfaces involve two things: “interface types” and “interface values”.

An interface type lists the methods a concrete type needs in order to be considered an instance of that interface type.

type car interface {
	accelerate(i int) error
	brake()           error
}

An interface type establishes a contract based on its methods, but does not specify an implementation for the methods. A concrete type that wants to fulfill the interface contract is responsible for implementing the methods. Anything that implements the methods of an interface type can be treated as that type. For example, a dunebuggy type that implements both accelerate and brake methods can be passed to an oilchange function that accepts car types. Go’s Fprintf, for example, outputs to anything with a Write method.

Statically typed languages, like Go, fix types at compile time. During compilation, for each type, Go generates a “type descriptor” that includes the type’s name and methods. An interface value holds a pair of pointers. One points to a particular concrete value (also called the interface value’s “dynamic value”). The other points to the concrete type’s matching “type descriptor” (this is also called the interface value’s “dynamic type”).

Basically, that’s it. Interfaces are fairly simple. However, the interface concept is flexible enough that Go uses it in many ways.

An interface defines a contract (required methods) without specifying an implementation:

type geometry interface {
	area() float64
	perim() float64
}

The interface lets us write a function that works on different types, so long as they all implement the interface’s methods:

type rect struct {
	width, height float64
}
type circle struct {
	radius float64
}
func (r rect) area() float64 {
	return r.width * r.height
}
func (r rect) perim() float64 {
	return 2*r.width + 2*r.height
}
func (c circle) area() float64 {
	return math.Pi * c.radius * c.radius
}
func (c circle) perim() float64 {
	return 2 * math.Pi * c.radius
}
func measure(g geometry) {
	fmt.Println(g)
	fmt.Println(g.area())
	fmt.Println(g.perim())
}
func main() {
	r := rect{width: 3, height: 4}
	c := circle{radius: 5}
	measure(r)
	measure(c)
}

In a loose analogy, if concrete types like int and string are nouns, and functions are verbs, then interfaces are adjectives. rect and circle are “geometry-ish”.

An interface value holds any value that implements the interface:

var x geometry
r2 := circle{radius: 9}
x = r2
x = rect{width:2, height:5}

Interfaces with only one or two methods are common in Go. The method often names the interface, like io.Writer for something that implements Write. A type can implement multiple interfaces.

A type that possesses all the methods of an interface implicitly satisfies its requirements. There’s no need to explicitly declare any of the interfaces associated with a type.

Because interfaces decouple the definition from the implementation, definitions can appear in one package and implementations in a different package.

Interfaces can be used as fields in structures, passed as arguments to functions, and so forth.

Interface Values and Nil

Think of interface values as a tuple of a value and an underlying concrete type:

(value, type)

If the concrete value is nil, a method called on the interface value will be called with a nil receiver. In some languages this triggers a null pointer exception, but Go methods can handle being called with a nil receiver.

func (t *T) M() {
	if t == nil {
		fmt.Println("<nil>")
		return
	}
	fmt.Println(t.S)
}

An interface value that holds a nil concrete value is not itself nil. A nil interface value holds neither value nor concrete type. Calling a method on a nil interface is a run-time error.

An interface value hides the underlying concrete value it holds. If the concrete value has a method that the interface does not, the interface prevents calling the hidden method. Values of type interface{} have zero exposed methods.

The Empty Interface

The interface type that specifies zero methods is known as the empty interface (interface{}). An empty interface may hold values of any type. (Every type implements at least zero methods.) Empty interfaces are used by code that handles values of unknown type. For example, fmt.Print takes any number of arguments of type interface{}.

func describe(i interface{}) {
	fmt.Printf("(%v, %T)\n", i, i)
}

func main() {
	var i interface{}
	describe(i) // (<nil>, <nil>)
	i = 42
	describe(i) // (42, int)
	i = "hello"
	describe(i) // (hello, string)
}

If we treat a string as the empty interface{} type, the underlying dynamic value still contains the actual string, and the dynamic type indicates its methods.

Type Assertions

A type assertion provides access to an interface value’s underlying concrete value.

t := x.(T)

This statement asserts that the interface value x holds an underlying value of type T and assigns the underlying T value to the variable t. If x does not hold a T, the statement will trigger a panic. To test whether an interface value holds a specific type, a type assertion can return two values: the underlying value and a boolean value that reports whether the assertion succeeded.

t, ok := x.(T)

If x holds a T, then t will be the underlying value and ok will be true. If not, ok will be false and t will be the zero value of type T, and no panic occurs. (Using t after a failed assertion panics, so check the value of ok first!)

What does Go actually do during the type assertion? If the asserted type T is a concrete type, Go checks if the dynamic type of x is identical to to T. If the assertion succeeds, Go extracts the concrete value from x and assigns it to t. If, on the other hand, T is an interface type, Go check if the dynamic type of x satisfies T. If it does, Go slaps an interface type T on top of the result, rather than extracting the value alone.

Note the similarity between this syntax and that of reading from a map.

Type switches

When we do a type switch, Go checks the type descriptor indicated by the dynamic type pointer to see if the type’s methods fulfill the contract of the type we’re testing/switching.

The type switch construct permits several type assertions in series. Unlike a regular switch statement, the cases in a type switch specify types (not values), and compare those types against the dynamic type of the interface value.

func do(i interface{}) {
	switch v := i.(type) {
	case int:
		fmt.Printf("Twice %v is %v\n", v, v*2)
	case string:
		fmt.Printf("%q is %v bytes long\n", v, len(v))
	default:
		fmt.Printf("I don't know about type %T!\n", v)
	}
}

func main() {
	do(21)
	do("hello")
	do(true)
}

The declaration in a type switch has the same syntax as a type assertion i.(T), but the keyword type replaces the specific type T.

This switch statement tests whether interface value i holds a value of type T or S. In each of the T and S cases, the variable v will be of type T or S respectively and hold the value held by i. In the default case (where there is no match), the variable v is of the same interface type and value as i.

Extending Types: Aliasing vs. Embedding

It’s tempting to imagine we can extend a type by doing something like this:

type ringLog ring.Ring

… but aliasing a type like this effectively creates a new type. Variables of type ringLog have access to members of ring.Ring, but not its methods. And there’s no good way to assert a ringLog into a ring.Ring to get to the methods.

It’s more useful to embed than alias. Embed the non-local type inside a local type as an anonymous struct field. This allows access to the non-local type’s methods.

type ringLog struct {
	*ring.Ring
}

func (r *ringLog) Write(p []byte) (int, error) {
	r.Ring.Value = string(p)
	r.Ring = r.Ring.Next()
	return len(p), nil
}

func main() {
	errRing := &ringLog{ring.New(5)}
	// …
}

https://play.golang.org/p/GUtb-kArrie

Interface Example: Stringer

package main

import "fmt"

type IPAddr [4]byte

func (a IPAddr) String() string {
	return fmt.Sprintf("%d.%d.%d.%d", a[0], a[1], a[2], a[3])
}

func main() {
	hosts := map[string]IPAddr{
		"loopback":  {127, 0, 0, 1},
		"googleDNS": {8, 8, 8, 8},
	}
	for name, ip := range hosts {
		fmt.Printf("%v: %v\n", name, ip)
	}
}

Interface Example: Error

package main

import (
	"fmt"
	"time"
)

type MyError struct {
	When time.Time
	What string
}

func (e *MyError) Error() string {
	return fmt.Sprintf("at %v, %s", e.When, e.What)
}

func run() error {
	return &MyError{
		time.Now(),
		"it didn't work",
	}
}

func main() {
	if err := run(); err != nil {
		fmt.Println(err)
	}
}

Interface Example: Readers

https://golang.org/pkg/io/#Reader

The io package specifies the io.Reader interface, which represents the read end of a stream of data:

func (T) Read(b []byte) (n int, err error)

Read populates the given byte slice with data, and returns the number of bytes populated along with an error value. It returns an io.EOF error when the stream ends.

In one common pattern, an io.Reader wraps another io.Reader, modifying the stream in some way. For example, the gzip.NewReader function takes an io.Reader (a stream of compressed data) and returns a *gzip.Reader that also implements io.Reader (a stream of the decompressed data).

package main

import (
	"io"
	"os"
	"strings"
)

type rot13Reader struct {
	rdr io.Reader
}

func (rot *rot13Reader) Read(p []byte) (n int, err error) {
	n, err = rot.rdr.Read(p) // We call the "wrapped" io.Reader's Read() to fill p.
	for i, letter := range p[:n] {
		switch {
		case letter >= 'A' && letter <= 'Z':
			p[i] = 'A' + (letter - 'A' + 13) % 26
		case letter >= 'a' && letter <= 'z':
			p[i] = 'a' + (letter - 'a' + 13) % 26
		}
	}
	return n, err
}

func main() {
	s := strings.NewReader("Lbh penpxrq gur pbqr!")
	r := rot13Reader{s}
	io.Copy(os.Stdout, &r)
}

Concurrency

Go provides concurrency using goroutines and channels.

An OS process has a PID and a block of memory. An OS thread is less expensive than forking an additional process; threads share the memory, open files, and PID of their parent (although each thread has a Thread ID).

A goroutine is like a thread scheduled by Go instead of by the OS. Goroutines have much lower overhead than OS threads. Go generally doesn’t start a new OS thread for a goroutine. The Go runtime holds a number of threads and multiplexes scheduling goroutines between them; if a goroutine blocks, Go swaps it out to let another goroutine execute on that thread. Goroutines are a lightweight abstraction on top of threads; a Go program doesn’t deal with treads, and the OS isn’t aware of goroutines.

Rob Pike’s talk about concurrency is instructive: https://www.youtube.com/watch?v=cN_DpYBzKso

Start a goroutine using the go keyword.

func work() {
	fmt.Println("Working...")
}

func main() {
	fmt.Println("Started.")
	go work()
	time.Sleep(time.Millisecond * 10)    // For demonstration purposes only!
	fmt.Println("Done.")
}

Because goroutines run in the same address space, synchronize access to shared memory. Concurrent reading may be safe, but certainly synchronization whenever concurrently writing values. For simple cases, a mutex may be enough (i.e. sync.Mutex).

However, channels provide an alternative to the complications of safely sharing memory. Go’s preference for channels is often states as:

Don’t communicate by sharing memory; share memory by communicating.

A channel is a bit like a pipe in a shell, but it has a type. The type matches the kind of data to be passed through the channel.

ch := make(chan int) // A channel for int data.
func worker(ch chan int) { … } // Pass the channel to a function as type "chan int".

Channels support sending and receiving, with direction indicated using arrow notation:

ch <- d // Send data to channel.
d := <-ch // Receive data from channel.

Note that sending data to and receiving data from a channel is blocking; execution of the goroutine pauses until data flows across the channel when the other side is ready. This allows goroutines to synchronize without explicit locks or condition variables.

This example spawns four goroutines to server forever, all reading data from the channel:

package main

import (
	"fmt"
	"time"
	"math/rand"
)

type Worker struct {
	id int
}

func (w *Worker) process(c chan int) {
	for {
		data := <-c
		fmt.Printf("worker %d got %d\n", w.id, data)
	}
}

func main() {
	c := make(chan int)
	for i := 0; i < 5; i++ {
		worker := &Worker{id: i}
		go worker.process(c)
	}
	for {
		c <- rand.Int()
		time.Sleep(time.Millisecond * 50)
	}
}

What if we have a bunch of of channels, and want to read from the one with pending data? Go’s select statement is like switch, but the decision is based on the ability to communicate rather than value tests. select lets a goroutine wait on multiple communication channels. If no channel is ready to communicate, select falls through to the default clause.

func fibonacci(c, quit chan int) {
	x, y := 0, 1
	for {
		select {
		case c <- x:
			x, y = y, x+y
		case <-quit:
			fmt.Println("quit")
			return
		}
	}
}

func main() {
	c := make(chan int)
	quit := make(chan int)
	go func() {
		for i := 0; i < 10; i++ {
			fmt.Println(<-c)
		}
		quit <- 0
	}()
	fibonacci(c, quit)
}

We can make a channel buffer. Sends to a buffered channel only blocks when the buffer fills; receives only block when its empty.

func main() {
	ch := make(chan int, 2)
	ch <- 1
	ch <- 2
	ch <- 3 // Sending too much to a buffered channel causes a fatal deadlock.
	fmt.Println(<-ch)
	fmt.Println(<-ch)
}

Sometimes channels send events (i.e., the occurrence of the signal is the only thing that’s important, not the data sent). Although we can send a tiny boolean for the signal, a Go idiom is to use a struct. The empty struct emphasizes the “event” nature of the channel.

func main() {
	// [Set up some stuff.]
	done := make(chan struct{})
	go func() {
		// [Do some stuff.]
		log.Println("done")
		done <- struct{}{} // Signal the main goroutine with an empty struct.
	}()
	// [Do other stuff.]
	<-done // Wait the background goroutine to end.
}

Range and Close

A sender can close a channel to indicate that no more values will be sent. Receivers test whether a channel has been closed by assigning a second parameter to the receive expression:

v, ok := <-ch

ok is false if there are no more values to receive and the channel is closed.

The loop for i := range c receives values from the channel repeatedly until it is closed.

Note: Only the sender should close a channel, never the receiver. Sending on a closed channel will cause a panic.

Another note: Channels aren’t like files; you don’t usually need to close them. Closing is only necessary when the receiver must be told there are no more values coming, such as to terminate a range loop.

package main

import "fmt"

func fibonacci(n int, c chan int) {
	x, y := 0, 1
	for i := 0; i < n; i++ {
		c <- x
		x, y = y, x+y
	}
	close(c)
}

func main() {
	c := make(chan int, 10)
	go fibonacci(cap(c), c)
	for i := range c {
		fmt.Println(i)
	}
}

sync.Mutex

Channels are great for communication among goroutines, but sometimes they need to share memory. Mutual exclusion is a simple method to avoid conflicts when multiple goroutines access a variable.

package main

import (
	"fmt"
	"sync"
	"time"
)

// SafeCounter is safe to use concurrently.
type SafeCounter struct {
	v   map[string]int
	mux sync.Mutex
}

// Inc increments the counter for the given key.
func (c *SafeCounter) Inc(key string) {
	c.mux.Lock()
	// Lock so only one goroutine at a time can access the map c.v.
	c.v[key]++
	c.mux.Unlock()
}

// Value returns the current value of the counter for the given key.
func (c *SafeCounter) Value(key string) int {
	c.mux.Lock()
	// Lock so only one goroutine at a time can access the map c.v.
	defer c.mux.Unlock()
	return c.v[key]
}

func main() {
	c := SafeCounter{v: make(map[string]int)}
	for i := 0; i < 1000; i++ {
		go c.Inc("somekey")
	}

	time.Sleep(time.Second)
	fmt.Println(c.Value("somekey"))
}

The `<<` and `>>` Operators (left shift and right shift)

https://golang.org/ref/spec#Arithmetic_operators

The shift operators shift the left operand by the shift count specified by the right operand. They implement arithmetic shifts if the left operand is a signed integer and logical shifts if it is an unsigned integer. There is no upper limit on the shift count. Shifts behave as if the left operand is shifted n times by 1 for a shift count of n. As a result, x << 1 is the same as x*2 and x >> 1 is the same as x/2 but truncated towards negative infinity.

https://stackoverflow.com/questions/5801008/go-and-operators

The super (possibly over) simplified definition is just that << is used for “times 2” and >> is for “divided by 2” - and the number after it is how many times. So n << x is “n times 2, x times”. And y >> z is “y divided by 2, z times”. For example, 1 << 5 is “1 times 2, 5 times” or 32. And 32 >> 5 is “32 divided by 2, 5 times” or 1.

Testing

Go includes a lightweight testing framework comprised of the testing package and the go test tool. Create a test file with a name ending in _test.go. The _test.go file should contain functions named TestFOO with the signature:

func (t *testing.T)

If the test function calls a failure function like t.Error or t.Fail, Go considers the test failed.

For a package named stringutil with a Reverse function, create a stringutil_test.go file:

package stringutil

import "testing"

func TestReverse(t *testing.T) {
	cases := []struct {
		in, want string
	}{
		{"Hello, world", "dlrow ,olleH"},
		{"Hello, 世界", "界世 ,olleH"},
		{"", ""},
	}
	for _, c := range cases {
		got := Reverse(c.in)
		if got != c.want {
			t.Errorf("Reverse(%q) == %q, want %q", c.in, got, c.want)
		}
	}
}

… and run the test:

$ go test example.com/stringutil

Init

Each source file may define an init function. Go calls init after initialization of global variable declarations and initialization of imported packages. Importing a package runs the package’s init function. A package can have multiple init function; they run in the order they appear in the code.

Apart from initialization that simple declarations can’t handle, init functions commonly verify or repair the correctness of program state before beginning real execution.

func init() {
	if user == "" {
		log.Fatal("$USER not set")
	}
	if home == "" {
		home = "/home/" + user
	}
	if gopath == "" {
		gopath = home + "/go"
	}
	// gopath may be overridden by --gopath flag on command line.
	flag.StringVar(&gopath, "gopath", gopath, "override default GOPATH")
}

Idiomatic Go

Check err values returned by function.
Return err from your function if appropriate.
Use mixedCase/MixedCase rather than using underscore_names.
Favor concision for names (and code in general) as much as is reasonable.
Include comments for consumption by go doc:
- Summarize a package with a comment immediately proceeding the package foo declaration in one of the package’s files.
- Describe all the exported things with a comment proceeding the declaration, like Foo processes bar records in alphabetical order..
- Write complete sentences that begin with the name of the thing being documented.
- https://blog.golang.org/godoc-documenting-go-code
Keep code as simple as possible.
If there’s a reasonable way to do so, minimize indentation levels.
The Go source itself (including the standard library) is an excellent style example.

Read https://github.com/golang/go/wiki/CodeReviewComments

Git pre-commit hooks

$ touch ~/repo/go/src/example.com/myproject/.git/hooks/pre-commit
$ chmod a+x ~/repo/go/src/example.com/myproject/.git/hooks/pre-commit
$ cat << 'EOF' > ~/repo/go/src/example.com/myproject/.git/hooks/pre-commit

#!/bin/sh go fmt ./… go vet ./… go test -cover ./… find -type f -name ‘.go’ -exec wc -l {} + find -type f -name ‘.go’ -exec sed ‘/^\s*$/d’ {} + | wc -l; echo ‘ ↳ total (non-blank) lines of code’ EOF

Building and Build Flags

When building binaries for distribution, we may want to strip debug symbols to reduce the binary size:

% go build -ldflags '-s -w'

To build a static binary, which can be useful when moving it to a musl system:

5 CGO_ENABLED=0 go build -a -installsuffix cgo

While go build compiles the executable and writes it to the same directory, go install does a little more. go install moves the compiled binary to $GOPATH/bin/, and does some caching of imported packages.

Cross-Compiling

$ GOOS=windows GOARCH=amd64 go build
$ GOOS=openbsd GOARCH=386 go build
$ GOOS=linux GOARCH=386 go build
$ GOOS=freebsd GOARCH=arm64 go build

Go Standard Library Packages

Go has a solid set of packages in its standard library. See https://golang.org/pkg/

Time

The time.Format function uses a reference time: Mon Jan 2 15:04:05 -0700 MST 2006 (i.e., 01/02 03:04:05PM ‘06 -0700, or 0 1 2 3 4 5 6 7).

Other Go Tooling

$  go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/paul/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
[…]

$  go list -m all
example.org/inventory
github.com/bvinc/go-sqlite-lite v0.5.0

$  go mod why -m github.com/bvinc/go-sqlite-lite
# github.com/bvinc/go-sqlite-lite
example.org/inventory
github.com/bvinc/go-sqlite-lite/sqlite3

$  go mod tidy
$  go mod verify

$  go clean -modcache

$  go doc -src strings.Replace

$  go test .          # Run all tests in the current directory
$  go test ./...      # Run all tests in the current directory and sub-directories
$  go test ./foo/bar  # Run all tests in the ./foo/bar directory
$  go test -race ./...
$  go test -cover ./...

$  go vet ./...

$  go fix ./...