548 lines
15 KiB
Markdown
548 lines
15 KiB
Markdown
---
|
|
title: Go's http package by example
|
|
description: >-
|
|
The basics of using, testing, and composing apps built using go's net/http
|
|
package.
|
|
---
|
|
|
|
Go's [http](http://golang.org/pkg/net/http/) package has turned into one of my
|
|
favorite things about the Go programming language. Initially it appears to be
|
|
somewhat complex, but in reality it can be broken down into a couple of simple
|
|
components that are extremely flexible in how they can be used. This guide will
|
|
cover the basic ideas behind the http package, as well as examples in using,
|
|
testing, and composing apps built with it.
|
|
|
|
This guide assumes you have some basic knowledge of what an interface in Go is,
|
|
and some idea of how HTTP works and what it can do.
|
|
|
|
## Handler
|
|
|
|
The building block of the entire http package is the `http.Handler` interface,
|
|
which is defined as follows:
|
|
|
|
```go
|
|
type Handler interface {
|
|
ServeHTTP(ResponseWriter, *Request)
|
|
}
|
|
```
|
|
|
|
Once implemented the `http.Handler` can be passed to `http.ListenAndServe`,
|
|
which will call the `ServeHTTP` method on every incoming request.
|
|
|
|
`http.Request` contains all relevant information about an incoming http request
|
|
which is being served by your `http.Handler`.
|
|
|
|
The `http.ResponseWriter` is the interface through which you can respond to the
|
|
request. It implements the `io.Writer` interface, so you can use methods like
|
|
`fmt.Fprintf` to write a formatted string as the response body, or ones like
|
|
`io.Copy` to write out the contents of a file (or any other `io.Reader`). The
|
|
response code can be set before you begin writing data using the `WriteHeader`
|
|
method.
|
|
|
|
Here's an example of an extremely simple http server:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"log"
|
|
"net/http"
|
|
)
|
|
|
|
type helloHandler struct{}
|
|
|
|
func (h helloHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|
fmt.Fprintf(w, "hello, you've hit %s\n", r.URL.Path)
|
|
}
|
|
|
|
func main() {
|
|
err := http.ListenAndServe(":9999", helloHandler{})
|
|
log.Fatal(err)
|
|
}
|
|
```
|
|
|
|
`http.ListenAndServe` serves requests using the handler, listening on the given
|
|
address:port. It will block unless it encounters an error listening, in which
|
|
case we `log.Fatal`.
|
|
|
|
Here's an example of using this handler with curl:
|
|
|
|
```
|
|
~ $ curl localhost:9999/foo/bar
|
|
hello, you've hit /foo/bar
|
|
```
|
|
|
|
|
|
## HandlerFunc
|
|
|
|
Often defining a full type to implement the `http.Handler` interface is a bit
|
|
overkill, especially for extremely simple `ServeHTTP` functions like the one
|
|
above. The `http` package provides a helper function, `http.HandlerFunc`, which
|
|
wraps a function which has the signature
|
|
`func(w http.ResponseWriter, r *http.Request)`, returning an `http.Handler`
|
|
which will call it in all cases.
|
|
|
|
The following behaves exactly like the previous example, but uses
|
|
`http.HandlerFunc` instead of defining a new type.
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"log"
|
|
"net/http"
|
|
)
|
|
|
|
func main() {
|
|
h := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
fmt.Fprintf(w, "hello, you've hit %s\n", r.URL.Path)
|
|
})
|
|
|
|
err := http.ListenAndServe(":9999", h)
|
|
log.Fatal(err)
|
|
}
|
|
```
|
|
|
|
## ServeMux
|
|
|
|
On their own, the previous examples don't seem all that useful. If we wanted to
|
|
have different behavior for different endpoints we would end up with having to
|
|
parse path strings as well as numerous `if` or `switch` statements. Luckily
|
|
we're provided with `http.ServeMux`, which does all of that for us. Here's an
|
|
example of it being used:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"log"
|
|
"net/http"
|
|
)
|
|
|
|
func main() {
|
|
h := http.NewServeMux()
|
|
|
|
h.HandleFunc("/foo", func(w http.ResponseWriter, r *http.Request) {
|
|
fmt.Fprintln(w, "Hello, you hit foo!")
|
|
})
|
|
|
|
h.HandleFunc("/bar", func(w http.ResponseWriter, r *http.Request) {
|
|
fmt.Fprintln(w, "Hello, you hit bar!")
|
|
})
|
|
|
|
h.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(404)
|
|
fmt.Fprintln(w, "You're lost, go home")
|
|
})
|
|
|
|
err := http.ListenAndServe(":9999", h)
|
|
log.Fatal(err)
|
|
}
|
|
```
|
|
|
|
The `http.ServeMux` is itself an `http.Handler`, so it can be passed into
|
|
`http.ListenAndServe`. When it receives a request it will check if the request's
|
|
path is prefixed by any of its known paths, choosing the longest prefix match it
|
|
can find. We use the `/` endpoint as a catch-all to catch any requests to
|
|
unknown endpoints. Here's some examples of it being used:
|
|
|
|
```
|
|
~ $ curl localhost:9999/foo
|
|
Hello, you hit foo!
|
|
|
|
~ $ curl localhost:9999/bar
|
|
Hello, you hit bar!
|
|
|
|
~ $ curl localhost:9999/baz
|
|
You're lost, go home
|
|
```
|
|
|
|
`http.ServeMux` has both `Handle` and `HandleFunc` methods. These do the same
|
|
thing, except that `Handle` takes in an `http.Handler` while `HandleFunc` merely
|
|
takes in a function, implicitly wrapping it just as `http.HandlerFunc` does.
|
|
|
|
### Other muxes
|
|
|
|
There are numerous replacements for `http.ServeMux` like
|
|
[gorilla/mux](http://www.gorillatoolkit.org/pkg/mux) which give you things like
|
|
automatically pulling variables out of paths, easily asserting what http methods
|
|
are allowed on an endpoint, and more. Most of these replacements will implement
|
|
`http.Handler` like `http.ServeMux` does, and accept `http.Handler`s as
|
|
arguments, and so are easy to use in conjunction with the rest of the things
|
|
I'm going to talk about in this post.
|
|
|
|
## Composability
|
|
|
|
When I say that the `http` package is composable I mean that it is very easy to
|
|
create re-usable pieces of code and glue them together into a new working
|
|
application. The `http.Handler` interface is the way all pieces communicate with
|
|
each other. Here's an example of where we use the same `http.Handler` to handle
|
|
multiple endpoints, each slightly differently:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"log"
|
|
"net/http"
|
|
)
|
|
|
|
type numberDumper int
|
|
|
|
func (n numberDumper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|
fmt.Fprintf(w, "Here's your number: %d\n", n)
|
|
}
|
|
|
|
func main() {
|
|
h := http.NewServeMux()
|
|
|
|
h.Handle("/one", numberDumper(1))
|
|
h.Handle("/two", numberDumper(2))
|
|
h.Handle("/three", numberDumper(3))
|
|
h.Handle("/four", numberDumper(4))
|
|
h.Handle("/five", numberDumper(5))
|
|
|
|
h.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(404)
|
|
fmt.Fprintln(w, "That's not a supported number!")
|
|
})
|
|
|
|
err := http.ListenAndServe(":9999", h)
|
|
log.Fatal(err)
|
|
}
|
|
```
|
|
|
|
`numberDumper` implements `http.Handler`, and can be passed into the
|
|
`http.ServeMux` multiple times to serve multiple endpoints. Here's it in action:
|
|
|
|
```
|
|
~ $ curl localhost:9999/one
|
|
Here's your number: 1
|
|
~ $ curl localhost:9999/five
|
|
Here's your number: 5
|
|
~ $ curl localhost:9999/bazillion
|
|
That's not a supported number!
|
|
```
|
|
|
|
## Testing
|
|
|
|
Testing http endpoints is extremely easy in Go, and doesn't even require you to
|
|
actually listen on any ports! The `httptest` package provides a few handy
|
|
utilities, including `NewRecorder` which implements `http.ResponseWriter` and
|
|
allows you to effectively make an http request by calling `ServeHTTP` directly.
|
|
Here's an example of a test for our previously implemented `numberDumper`,
|
|
commented with what exactly is happening:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
. "testing"
|
|
)
|
|
|
|
func TestNumberDumper(t *T) {
|
|
// We first create the http.Handler we wish to test
|
|
n := numberDumper(1)
|
|
|
|
// We create an http.Request object to test with. The http.Request is
|
|
// totally customizable in every way that a real-life http request is, so
|
|
// even the most intricate behavior can be tested
|
|
r, _ := http.NewRequest("GET", "/one", nil)
|
|
|
|
// httptest.Recorder implements the http.ResponseWriter interface, and as
|
|
// such can be passed into ServeHTTP to receive the response. It will act as
|
|
// if all data being given to it is being sent to a real client, when in
|
|
// reality it's being buffered for later observation
|
|
w := httptest.NewRecorder()
|
|
|
|
// Pass in our httptest.Recorder and http.Request to our numberDumper. At
|
|
// this point the numberDumper will act just as if it was responding to a
|
|
// real request
|
|
n.ServeHTTP(w, r)
|
|
|
|
// httptest.Recorder gives a number of fields and methods which can be used
|
|
// to observe the response made to our request. Here we check the response
|
|
// code
|
|
if w.Code != 200 {
|
|
t.Fatalf("wrong code returned: %d", w.Code)
|
|
}
|
|
|
|
// We can also get the full body out of the httptest.Recorder, and check
|
|
// that its contents are what we expect
|
|
body := w.Body.String()
|
|
if body != fmt.Sprintf("Here's your number: 1\n") {
|
|
t.Fatalf("wrong body returned: %s", body)
|
|
}
|
|
|
|
}
|
|
```
|
|
|
|
In this way it's easy to create tests for your individual components that you
|
|
are using to build your application, keeping the tests near to the functionality
|
|
they're testing.
|
|
|
|
Note: if you ever do need to spin up a test server in your tests, `httptest`
|
|
also provides a way to create a server listening on a random open port for use
|
|
in tests as well.
|
|
|
|
## Middleware
|
|
|
|
Serving endpoints is nice, but often there's functionality you need to run for
|
|
*every* request before the actual endpoint's handler is run. For example, access
|
|
logging. A middleware component is one which implements `http.Handler`, but will
|
|
actually pass the request off to another `http.Handler` after doing some set of
|
|
actions. The `http.ServeMux` we looked at earlier is actually an example of
|
|
middleware, since it passes the request off to another `http.Handler` for actual
|
|
processing. Here's an example of our previous example with some logging
|
|
middleware:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"log"
|
|
"net/http"
|
|
)
|
|
|
|
type numberDumper int
|
|
|
|
func (n numberDumper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|
fmt.Fprintf(w, "Here's your number: %d\n", n)
|
|
}
|
|
|
|
func logger(h http.Handler) http.Handler {
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
log.Printf("%s requested %s", r.RemoteAddr, r.URL)
|
|
h.ServeHTTP(w, r)
|
|
})
|
|
}
|
|
|
|
func main() {
|
|
h := http.NewServeMux()
|
|
|
|
h.Handle("/one", numberDumper(1))
|
|
h.Handle("/two", numberDumper(2))
|
|
h.Handle("/three", numberDumper(3))
|
|
h.Handle("/four", numberDumper(4))
|
|
h.Handle("/five", numberDumper(5))
|
|
|
|
h.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(404)
|
|
fmt.Fprintln(w, "That's not a supported number!")
|
|
})
|
|
|
|
hl := logger(h)
|
|
|
|
err := http.ListenAndServe(":9999", hl)
|
|
log.Fatal(err)
|
|
}
|
|
```
|
|
|
|
`logger` is a function which takes in an `http.Handler` called `h`, and returns
|
|
a new `http.Handler` which, when called, will log the request it was called with
|
|
and then pass off its arguments to `h`. To use it we pass in our
|
|
`http.ServeMux`, so all incoming requests will first be handled by the logging
|
|
middleware before being passed to the `http.ServeMux`.
|
|
|
|
Here's an example log entry which is output when the `/five` endpoint is hit:
|
|
|
|
```
|
|
2015/06/30 20:15:41 [::1]:34688 requested /five
|
|
```
|
|
|
|
## Middleware chaining
|
|
|
|
Being able to chain middleware together is an incredibly useful ability which we
|
|
get almost for free, as long as we use the signature
|
|
`func(http.Handler) http.Handler`. A middleware component returns the same type
|
|
which is passed into it, so simply passing the output of one middleware
|
|
component into the other is sufficient.
|
|
|
|
However, more complex behavior with middleware can be tricky. For instance, what
|
|
if you want a piece of middleware which takes in a parameter upon creation?
|
|
Here's an example of just that, with a piece of middleware which will set a
|
|
header and its value for all requests:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"log"
|
|
"net/http"
|
|
)
|
|
|
|
type numberDumper int
|
|
|
|
func (n numberDumper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|
fmt.Fprintf(w, "Here's your number: %d\n", n)
|
|
}
|
|
|
|
func logger(h http.Handler) http.Handler {
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
log.Printf("%s requested %s", r.RemoteAddr, r.URL)
|
|
h.ServeHTTP(w, r)
|
|
})
|
|
}
|
|
|
|
type headerSetter struct {
|
|
key, val string
|
|
handler http.Handler
|
|
}
|
|
|
|
func (hs headerSetter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|
w.Header().Set(hs.key, hs.val)
|
|
hs.handler.ServeHTTP(w, r)
|
|
}
|
|
|
|
func newHeaderSetter(key, val string) func(http.Handler) http.Handler {
|
|
return func(h http.Handler) http.Handler {
|
|
return headerSetter{key, val, h}
|
|
}
|
|
}
|
|
|
|
func main() {
|
|
h := http.NewServeMux()
|
|
|
|
h.Handle("/one", numberDumper(1))
|
|
h.Handle("/two", numberDumper(2))
|
|
h.Handle("/three", numberDumper(3))
|
|
h.Handle("/four", numberDumper(4))
|
|
h.Handle("/five", numberDumper(5))
|
|
|
|
h.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(404)
|
|
fmt.Fprintln(w, "That's not a supported number!")
|
|
})
|
|
|
|
hl := logger(h)
|
|
hhs := newHeaderSetter("X-FOO", "BAR")(hl)
|
|
|
|
err := http.ListenAndServe(":9999", hhs)
|
|
log.Fatal(err)
|
|
}
|
|
```
|
|
|
|
And here's the curl output:
|
|
|
|
```
|
|
~ $ curl -i localhost:9999/three
|
|
HTTP/1.1 200 OK
|
|
X-Foo: BAR
|
|
Date: Wed, 01 Jul 2015 00:39:48 GMT
|
|
Content-Length: 22
|
|
Content-Type: text/plain; charset=utf-8
|
|
|
|
Here's your number: 3
|
|
|
|
```
|
|
|
|
`newHeaderSetter` returns a function which accepts and returns an
|
|
`http.Handler`. Calling that returned function with an `http.Handler` then gets
|
|
you an `http.Handler` which will set the header given to `newHeaderSetter`
|
|
before continuing on to the given `http.Handler`.
|
|
|
|
This may seem like a strange way of organizing this; for this example the
|
|
signature for `newHeaderSetter` could very well have looked like this:
|
|
|
|
```
|
|
func newHeaderSetter(key, val string, h http.Handler) http.Handler
|
|
```
|
|
|
|
And that implementation would have worked fine. But it would have been more
|
|
difficult to compose going forward. In the next section I'll show what I mean.
|
|
|
|
## Composing middleware with alice
|
|
|
|
[Alice](https://github.com/justinas/alice) is a very simple and convenient
|
|
helper for working with middleware using the function signature we've been using
|
|
thusfar. Alice is used to create and use chains of middleware. Chains can even
|
|
be appended to each other, giving even further flexibility. Here's our previous
|
|
example with a couple more headers being set, but also using alice to manage the
|
|
added complexity.
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"log"
|
|
"net/http"
|
|
|
|
"github.com/justinas/alice"
|
|
)
|
|
|
|
type numberDumper int
|
|
|
|
func (n numberDumper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|
fmt.Fprintf(w, "Here's your number: %d\n", n)
|
|
}
|
|
|
|
func logger(h http.Handler) http.Handler {
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
log.Printf("%s requested %s", r.RemoteAddr, r.URL)
|
|
h.ServeHTTP(w, r)
|
|
})
|
|
}
|
|
|
|
type headerSetter struct {
|
|
key, val string
|
|
handler http.Handler
|
|
}
|
|
|
|
func (hs headerSetter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|
w.Header().Set(hs.key, hs.val)
|
|
hs.handler.ServeHTTP(w, r)
|
|
}
|
|
|
|
func newHeaderSetter(key, val string) func(http.Handler) http.Handler {
|
|
return func(h http.Handler) http.Handler {
|
|
return headerSetter{key, val, h}
|
|
}
|
|
}
|
|
|
|
func main() {
|
|
h := http.NewServeMux()
|
|
|
|
h.Handle("/one", numberDumper(1))
|
|
h.Handle("/two", numberDumper(2))
|
|
h.Handle("/three", numberDumper(3))
|
|
h.Handle("/four", numberDumper(4))
|
|
|
|
fiveHS := newHeaderSetter("X-FIVE", "the best number")
|
|
h.Handle("/five", fiveHS(numberDumper(5)))
|
|
|
|
h.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
|
w.WriteHeader(404)
|
|
fmt.Fprintln(w, "That's not a supported number!")
|
|
})
|
|
|
|
chain := alice.New(
|
|
newHeaderSetter("X-FOO", "BAR"),
|
|
newHeaderSetter("X-BAZ", "BUZ"),
|
|
logger,
|
|
).Then(h)
|
|
|
|
err := http.ListenAndServe(":9999", chain)
|
|
log.Fatal(err)
|
|
}
|
|
```
|
|
|
|
In this example all requests will have the headers `X-FOO` and `X-BAZ` set, but
|
|
the `/five` endpoint will *also* have the `X-FIVE` header set.
|
|
|
|
## Fin
|
|
|
|
Starting with a simple idea of an interface, the `http` package allows us to
|
|
create for ourselves an incredibly useful and flexible (yet still rather simple)
|
|
ecosystem for building web apps with re-usable components, all without breaking
|
|
our static checks.
|