Go is an amazing language with a lot of momentum, and it's focused on simplicity. This approach is evident in its standard library, which provides all the essentials, but not much more.
Fortunately, Go has a vibrant community that creates and shares a lot of third-party libraries. In this tutorial, I'll introduce you to 12 of Go's best packages and libraries. Some of them have relatively narrow scope and can be added to any projects, while others are huge projects that you can incorporate into massive, large-scale distributed systems.
Awesome Go
Before diving into the libraries themselves, let me introduce you to Awesome Go, a very active and curated list of Go libraries and other resources. You should visit every now and then and check what's new.
1. Golang-Set
Go has arrays, slices and maps, but it doesn't have a set data structure. You can mimic a set with a map of bools, but it's nice to have an actual data type with the right operations and semantics. This is where golang-set comes in. Here is a basic example of creating a new set, adding items and testing for membership:
package main
import (
"fmt"
"http://ift.tt/1D7aMVH"
)
func main() {
basicColors := mapset.NewSet()
basicColors.Add("Red")
basicColors.Add("Blue")
basicColors.Add("Green")
if basicColors.Contains("Green") {
fmt.Println("Yay! 'Green' is a basic color")
} else {
fmt.Println("What a disappointment! 'Green' is not a basic color")
}
if basicColors.Contains("Yellow") {
fmt.Println("Yay! 'Yellow' is a basic color")
} else {
fmt.Println("What a disappointment! 'Yellow' is not a basic color")
}
}
Output:
Yay! 'Green' is a basic color
What a disappointment! 'Yellow' is not a basic color
Note that the package name is "mapset". In addition to the basics, you perform all set operations like union, intersection, and difference. You can also iterate over the set values:
package main
import (
"fmt"
"http://ift.tt/1D7aMVH"
)
func main() {
basicColors := mapset.NewSet()
basicColors.Add("Red")
basicColors.Add("Blue")
basicColors.Add("Green")
otherColors := mapset.NewSetFromSlice([]interface{}{"Orange", "Yellow", "Indigo", "Violet"})
rainbowColors := basicColors.Union(otherColors)
for color := range rainbowColors.Iterator().C {
fmt.Println(color)
}
}
2. Color
Let's continue with the color theme. When writing command-line programs, it is useful to use colors to highlight important messages or distinguish between errors, successes, and warnings.
The color package gives an easy way to add some color to your programs (see what I did there?). It uses ANSII escape codes and supports Windows too! Here is a quick example:
package main
import (
"http://ift.tt/1hbWsY2"
)
func main() {
color.Red("Roses are red")
color.Blue("Violets are blue")
}
The color package supports mixing colors with background colors, styles like bold or italic, and sprinkling color with non-color output.
package main
import (
"http://ift.tt/1hbWsY2"
"fmt"
)
func main() {
minion := color.New(color.FgBlack).Add(color.BgYellow).Add(color.Bold)
minion.Println("Minion says: banana!!!!!!")
m := minion.PrintlnFunc()
m("I want another banana!!!!!")
slantedRed := color.New(color.FgRed, color.BgWhite, color.Italic).SprintFunc()
fmt.Println("I've made a huge", slantedRed("mistake"))
}
The color package has other useful features. Go ahead and explore more.
3. Now
Now is a very simple package that provides a convenience wrapper for the standard time package and makes it easy to work with various date and time constructs around the current time.
For example, you can get the beginning of the current minute or the end of the Sunday closest to the current time. Here is how to use "now":
package main
import (
"http://ift.tt/2vdTawg"
"fmt"
)
func main() {
fmt.Println("All the beginnings...")
fmt.Println(now.BeginningOfMinute())
fmt.Println(now.BeginningOfHour())
fmt.Println(now.BeginningOfDay())
fmt.Println(now.BeginningOfWeek())
fmt.Println(now.BeginningOfMonth())
fmt.Println(now.BeginningOfQuarter())
fmt.Println(now.BeginningOfYear())
}
Output:
All the beginnings...
2017-06-04 16:59:00 -0700 PDT
2017-06-04 16:00:00 -0700 PDT
2017-06-04 00:00:00 -0700 PDT
2017-06-04 00:00:00 -0700 PDT
2017-06-01 00:00:00 -0700 PDT
2017-04-01 00:00:00 -0700 PDT
2016-12-31 23:00:00 -0800 PST
You can also parse times and even add your own formats (which will require updating the known formats). The Now type embeds time.Time, so you can use all of the time.Time methods directly on Now objects.
4. Gen
The gen tool generates code for you—in particular, type-aware code that tries to alleviate the gap of not having templates or generics in Go.
You annotate your types with a special comment, and gen generates source files that you include in your project. No runtime magic. Let's see an example. Here is an annotated type.
// +gen slice:"Where,Count,GroupBy[int]"
type Person struct {
Name string
Age int
}
Running gen (make sure it's in your path) generates person_slice.go:
// Generated by: gen
// TypeWriter: slice
// Directive: +gen on Person
package main
// PersonSlice is a slice of type Person. Use it where you would use []Person.
type PersonSlice []Person
// Where returns a new PersonSlice whose elements return true for func. See: http://ift.tt/2vdZgN1
func (rcv PersonSlice) Where(fn func(Person) bool) (result PersonSlice) {
for _, v := range rcv {
if fn(v) {
result = append(result, v)
}
}
return result
}
// Count gives the number elements of PersonSlice that return true for the passed func. See: http://ift.tt/2vx3P7L
func (rcv PersonSlice) Count(fn func(Person) bool) (result int) {
for _, v := range rcv {
if fn(v) {
result++
}
}
return
}
// GroupByInt groups elements into a map keyed by int. See: http://ift.tt/2vejpCs
func (rcv PersonSlice) GroupByInt(fn func(Person) int) map[int]PersonSlice {
result := make(map[int]PersonSlice)
for _, v := range rcv {
key := fn(v)
result[key] = append(result[key], v)
}
return result
}
The code provides LINQ-like methods to operate on the PersonSlice type. It's simple to understand and nicely documented.
Here is how you use it. In the main function, a PersonSlice is defined. The age() function selects the age field from its Person argument. The generated GroupByInt() function takes the age() function and returns the people from the slice grouped by their age (34 is just Jim, but 23 has both Jane and Kyle).
package main
import (
"fmt"
)
// +gen slice:"Where,Count,GroupBy[int]"
type Person struct {
Name string
Age int
}
func age(p Person) int {
return p.Age
}
func main() {
people := PersonSlice {
{"Jim", 34},
{"Jane", 23},
{"Kyle", 23},
}
groupedByAge := people.GroupByInt(age)
fmt.Println(groupedByAge)
}
Output:
map[34:[{Jim 34}] 23:[{Jane 23} {Kyle 23}]]
5. Gorm
Go is known for its spartan nature. Database programming is no different. Most popular DB libraries for Go are pretty low-level. Gorm brings the world of object-relational mapping to Go with the following features:
- Associations (Has One, Has Many, Belongs To, Many To Many, Polymorphism)
- Callbacks (Before/After Create/Save/Update/Delete/Find)
- Preloading (eager loading)
- Transactions
- Composite Primary Key
- SQL Builder
- Auto Migrations
- Logger
- Extendable, write Plugins based on GORM callbacks
But it doesn't cover everything. If you come from Python, don't expect SQLAlchemy magic. For more fancy stuff, you'll have to go a lower level. Here is an example of how to use Gorm with sqlite. Note the embedded gorm.Model in the Product struct.
package main
import (
"http://ift.tt/1fhdUsj"
_ "http://ift.tt/2slwwk4"
)
type Product struct {
gorm.Model
Code string
Price uint
}
func main() {
db, err := gorm.Open("sqlite3", "test.db")
if err != nil {
panic("failed to connect database")
}
defer db.Close()
// Migrate the schema
db.AutoMigrate(&Product{})
// Create
db.Create(&Product{Code: "L1212", Price: 1000})
// Read
var product Product
db.First(&product, 1) // find product with id 1
db.First(&product, "code = ?", "L1212")
// Update - update product's price to 2000
db.Model(&product).Update("Price", 2000)
// Delete - delete product
db.Delete(&product)
6. Goose
One of the most important tasks when working with relational databases is managing the schema. Modifying the DB schema is considered a "scary" change in some organizations. The goose package lets you perform schema changes and even data migrations if needed. You can goose up and goose down to go back and forth. Mind your data, though, and make sure it doesn't get lost or corrupted.
Goose works by versioning your schema and using migration files corresponding to each schema. The migration files can be SQL commands or Go commands. Here is an example of a SQL migration file that adds a new table:
-- +goose Up
CREATE TABLE person (
id int NOT NULL,
name text,
age int,
PRIMARY KEY(id)
);
-- +goose Down
DROP TABLE person;
The -- +goose up and -- +goose down comments tell goose what to do to upgrade or downgrade the schema.
7. Glide
Glide is a package manager for Go. Under a single GOPATH, you may have many programs that have conflicting dependencies. The solution is to have each program manage its own vendor directory of package dependencies. Glide helps with this task.
Here are the features of glide:
- Support versioning packages including Semantic Versioning 2.0.0 support.
- Support aliasing packages (e.g. for working with github forks).
- Remove the need for munging import statements.
- Work with all of the go tools.
- Support all the VCS tools that Go supports (git, bzr, hg, svn).
- Support custom local and global plugins.
- Repository caching and data caching for improved performance.
- Flatten dependencies, resolving version differences and avoiding the inclusion of a package multiple times.
- Manage and install dependencies on-demand or vendored in your version control system.
The dependencies are stored in glide.yaml, and glide provides several commands to manage dependencies:
create, init Initialize a new project, creating a
glide.yaml file
config-wizard, cw Wizard that makes optional suggestions
to improve config in a glide.yaml file.
get Install one or more packages into
`vendor/` and add dependency to
glide.yaml.
remove, rm Remove a package from the glide.yaml
file, and regenerate the lock file.
import Import files from other dependency
management systems.
name Print the name of this project.
novendor, nv List all non-vendor paths in a
directory.
rebuild Rebuild ('go build') the dependencies
install, i Install a project's dependencies
update, up Update a project's dependencies
tree (Deprecated) Tree prints the
dependencies of this project as a tree.
list List prints all dependencies that the
present code references.
info Info prints information about this
project
cache-clear, cc Clears the Glide cache.
about Learn about Glide
mirror Manage mirrors
help, h Shows a list of commands or help for
one command
8. Ginkgo
Ginkgo is a BDD (Behavior Driven Development) testing framework. It lets you write your tests in a syntax that resembles English and allow less technical people to review tests (and their output) and verify that they match the business requirements.
Some developers like this style of test specification too. It integrates with Go's built-in testing package and is often combined with Gomega. Here is an example of a Ginkgo + Gomega test:
actual, err := foo() Ω(err).Should(BeNil()) Ω(actual).ShouldNot(BeNil()) Ω(actual.result).Should(Equal(100))
9. Etcd
Etcd is a reliable distributed Key-Value store. The server is implemented in Go, and the Go client interacts with it though gRPC.
It focuses on the following:
- Simple: well-defined, user-facing API (gRPC).
- Secure: automatic TLS with optional client cert authentication.
- Fast: benchmarked 10,000 writes/sec.
- Reliable: properly distributed using Raft.
Here is an example of connecting to the server, putting a value and getting it, including timeouts and cleanup.
func test_get() {
cli, err := clientv3.New(clientv3.Config{
Endpoints: endpoints,
DialTimeout: dialTimeout,
})
if err != nil {
log.Fatal(err)
}
defer cli.Close()
_, err = cli.Put(context.TODO(), "foo", "bar")
if err != nil {
log.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(),
requestTimeout)
resp, err := cli.Get(ctx, "foo")
cancel()
if err != nil {
log.Fatal(err)
}
for _, ev := range resp.Kvs {
fmt.Printf("%s : %s\n", ev.Key, ev.Value)
}
// Output: foo : bar
}
10. NSQ
NSQ is a great distributed queue. I've used it successfully as a primary building block for large-scale distributed systems. Here are some of its features:
- Support distributed topologies with no SPOF.
- Horizontally scalable (no brokers, seamlessly add more nodes to the cluster).
- Low-latency push based message delivery (performance).
- Combination load-balanced and multicast style message routing.
- Excel at both streaming (high-throughput) and job oriented (low-throughput) workloads.
- Primarily in-memory (beyond a high-water mark messages are transparently kept on disk).
- Runtime discovery service for consumers to find producers (nsqlookupd).
- Transport layer security (TLS).
- Data format agnostic.
- Few dependencies (easy to deploy) and a sane, bounded, default configuration.
- Simple TCP protocol supporting client libraries in any language.
- HTTP interface for stats, admin actions, and producers (no client library needed to publish).
- Integrates with statsd for real-time instrumentation.
- Robust cluster administration interface (nsqadmin).
Here is how to publish a message to NSQ (error handling is elided):
package main
import (
"http://ift.tt/1EKdTam"
)
func main() {
config := nsq.NewConfig()
p, _ := nsq.NewProducer("127.0.0.1:4150", config)
p.Publish("topic", []byte("message"))
p.Stop()
}
And here is how to consume:
package main
import (
"sync"
"fmt"
"http://ift.tt/1EKdTam"
)
func main() {
wg := &sync.WaitGroup{}
wg.Add(1)
config := nsq.NewConfig()
q, _ := nsq.NewConsumer("topic", "channel", config)
handler := nsq.HandlerFunc(func(message *nsq.Message) error {
fmt.Printf("Got a message: %v", message)
wg.Done()
return nil
})
q.AddHandler(handler)
q.ConnectToNSQD("127.0.0.1:4150")
wg.Wait()
}
11. Docker
Docker is a household name now (if your family members are mostly DevOps people). You may not be aware that Docker is implemented in Go. You don't typically use Docker in your code, but it is a significant project and deserves to be recognized as a hugely successful and popular Go project.
12. Kubernetes
Kubernetes is an open-source container orchestration platform for cloud-native applications. It is another monster distributed system implemented in Go. I recently wrote a book called Mastering Kubernetes where I go in detail over the most advanced aspects of Kubernetes. From the Go developer's point of view, Kubernetes is very flexible, and you can extend and customize it via plugins.
Conclusion
Go is a great language. Its design philosophy is to be a simple and approachable language. Its standard library is not as comprehensive as some other languages like Python.
The Go community stepped up, and there are many high-quality libraries you can use in your programs. In this article, I introduced 12 libraries. I encourage you to look for other libraries before jumping in and implementing everything from scratch.
No comments:
Post a Comment