Skip to content

GitLab

Commit 5dd74175 authored by Andrew Gerrand's avatar Andrew Gerrand
Browse files

doc: re-organize golang.org site content 

Remove "References" section.
Remove most articles and redirect to blog.golang.org.
Move /ref/spec and /ref/mem to /doc/spec and /doc/mem.
Remove duplicate links from the remaining
"Documents", "The Project", and "Help" pages.
Defer to the wiki for more links and community content.
Update command reference and mention cover tool.
Add "Pop-out" text to the front page.
Pick one of four videos at random to feature on the front page.

Fixes #2547.
Fixes #5561.
Fixes #6321.

R=r, dominik.honnef
CC=golang-dev
https://golang.org/cl/13724043
parent 83da0fd9
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 1803 deletions
doc/articles/c_go_cgo.html deleted 100644 → 0
View file @ 83da0fd9
<!--{
"Title": "C? Go? Cgo!",
"Template": true
}-->
<p>
Cgo lets Go packages call C code. Given a Go source file written with some
special features, cgo outputs Go and C files that can be combined into a
single Go package.
</p>
<p>
To lead with an example, here's a Go package that provides two functions -
<code>Random</code> and <code>Seed</code> - that wrap C's <code>rand</code>
and <code>srand</code> functions.
</p>
{{code "/doc/progs/cgo1.go" `/package rand/` `/END/`}}
<p>
Let's look at what's happening here, starting with the import statement.
</p>
<p>
The <code>rand</code> package imports <code>"C"</code>, but you'll find there's
no such package in the standard Go library. That's because <code>C</code> is a
"pseudo-package", a special name interpreted by cgo as a reference to C's
name space.
</p>
<p>
The <code>rand</code> package contains four references to the <code>C</code>
package: the calls to <code>C.rand</code> and <code>C.srand</code>, the
conversion <code>C.uint(i)</code>, and the <code>import</code> statement.
</p>
<p>
The <code>Random</code> function calls the standard C library's <code>random</code>
function and returns the result. In C, <code>rand</code> returns a value of the
C type <code>int</code>, which cgo represents as the type <code>C.int</code>.
It must be converted to a Go type before it can be used by Go code outside this
package, using an ordinary Go type conversion:
</p>
{{code "/doc/progs/cgo1.go" `/func Random/` `/STOP/`}}
<p>
Here's an equivalent function that uses a temporary variable to illustrate
the type conversion more explicitly:
</p>
{{code "/doc/progs/cgo2.go" `/func Random/` `/STOP/`}}
<p>
The <code>Seed</code> function does the reverse, in a way. It takes a
regular Go <code>int</code>, converts it to the C <code>unsigned int</code>
type, and passes it to the C function <code>srand</code>.
</p>
{{code "/doc/progs/cgo1.go" `/func Seed/` `/END/`}}
<p>
Note that cgo knows the <code>unsigned int</code> type as <code>C.uint</code>;
see the <a href="/cmd/cgo">cgo documentation</a> for a complete list of
these numeric type names.
</p>
<p>
The one detail of this example we haven't examined yet is the comment
above the <code>import</code> statement.
</p>
{{code "/doc/progs/cgo1.go" `/\/\*/` `/STOP/`}}
<p>
Cgo recognizes this comment. Any lines starting
with <code>#cgo</code>
followed
by a space character are removed; these become directives for cgo.
The remaining lines are used as a header when compiling the C parts of
the package. In this case those lines are just a
single <code>#include</code>
statement, but they can be almost any C code. The <code>#cgo</code>
directives are
used to provide flags for the compiler and linker when building the C
parts of the package.
</p>
<p>
There is a limitation: if your program uses any <code>//export</code>
directives, then the C code in the comment may only include declarations
(<code>extern int f();</code>), not definitions (<code>int f() {
return 1; }</code>). You can use <code>//export</code> directives to
make Go functions accessible to C code.
</p>
<p>
The <code>#cgo</code> and <code>//export</code> directives are
documented in
the <a href="/cmd/cgo/">cgo documentation</a>.
</p>
<p>
<b>Strings and things</b>
</p>
<p>
Unlike Go, C doesn't have an explicit string type. Strings in C are
represented by a zero-terminated array of chars.
</p>
<p>
Conversion between Go and C strings is done with the
<code>C.CString</code>, <code>C.GoString</code>, and
<code>C.GoStringN</code> functions. These conversions make a copy of the
string data.
</p>
<p>
This next example implements a <code>Print</code> function that writes a
string to standard output using C's <code>fputs</code> function from the
<code>stdio</code> library:
</p>
{{code "/doc/progs/cgo3.go" `/package print/` `/END/`}}
<p>
Memory allocations made by C code are not known to Go's memory manager.
When you create a C string with <code>C.CString</code> (or any C memory
allocation) you must remember to free the memory when you're done with it
by calling <code>C.free</code>.
</p>
<p>
The call to <code>C.CString</code> returns a pointer to the start of the
char array, so before the function exits we convert it to an
<a href="/pkg/unsafe/#Pointer"><code>unsafe.Pointer</code></a> and release
the memory allocation with <code>C.free</code>. A common idiom in cgo programs
is to <a href="/doc/articles/defer_panic_recover.html"><code>defer</code></a>
the free immediately after allocating (especially when the code that follows
is more complex than a single function call), as in this rewrite of
<code>Print</code>:
</p>
{{code "/doc/progs/cgo4.go" `/func Print/` `/END/`}}
<p>
<b>Building cgo packages</b>
</p>
<p>
To build cgo packages, just use <a href="/cmd/go/#hdr-Compile_packages_and_dependencies">"
<code>go build</code>"</a> or
<a href="/cmd/go/#hdr-Compile_and_install_packages_and_dependencies">"<code>go install</code>
"</a> as usual. The go tool recognizes the special <code>"C"</code> import and automatically
uses cgo for those files.
</p>
<p>
<b>More cgo resources</b>
</p>
<p>
The <a href="/cmd/cgo/">cgo command</a> documentation has more detail about
the C pseudo-package and the build process. The <a href="/misc/cgo/">cgo examples</a>
in the Go tree demonstrate more advanced concepts.
</p>
<p>
For a simple, idiomatic example of a cgo-based package, see Russ Cox's <a
href="http://code.google.com/p/gosqlite/source/browse/sqlite/sqlite.go">gosqlite</a>.
Also, the <a href="http://code.google.com/p/go-wiki/wiki/Projects">Go Community Wiki</a>
lists many packages, some of which use cgo.
</p>
<p>
Finally, if you're curious as to how all this works internally, take a look
at the introductory comment of the runtime package's <a href="/src/pkg/runtime/cgocall.c">cgocall.c</a>.
</p>
doc/articles/concurrency_patterns.html deleted 100644 → 0
View file @ 83da0fd9
<!--{
"Title": "Go Concurrency Patterns: Timing out, moving on",
"Template": true
}-->
<p>
Concurrent programming has its own idioms. A good example is timeouts. Although
Go's channels do not support them directly, they are easy to implement. Say we
want to receive from the channel <code>ch</code>, but want to wait at most one
second for the value to arrive. We would start by creating a signalling channel
and launching a goroutine that sleeps before sending on the channel:
</p>
{{code "/doc/progs/timeout1.go" `/timeout :=/` `/STOP/`}}
<p>
We can then use a <code>select</code> statement to receive from either
<code>ch</code> or <code>timeout</code>. If nothing arrives on <code>ch</code>
after one second, the timeout case is selected and the attempt to read from
<code>ch</code> is abandoned.
</p>
{{code "/doc/progs/timeout1.go" `/select {/` `/STOP/`}}
<p>
The <code>timeout</code> channel is buffered with space for 1 value, allowing
the timeout goroutine to send to the channel and then exit. The goroutine
doesn't know (or care) whether the value is received. This means the goroutine
won't hang around forever if the <code>ch</code> receive happens before the
timeout is reached. The <code>timeout</code> channel will eventually be
deallocated by the garbage collector.
</p>
<p>
(In this example we used <code>time.Sleep</code> to demonstrate the mechanics
of goroutines and channels. In real programs you should use <code>
<a href="/pkg/time/#After">time.After</a></code>, a function that returns
a channel and sends on that channel after the specified duration.)
</p>
<p>
Let's look at another variation of this pattern. In this example we have a
program that reads from multiple replicated databases simultaneously. The
program needs only one of the answers, and it should accept the answer that
arrives first.
</p>
<p>
The function <code>Query</code> takes a slice of database connections and a
<code>query</code> string. It queries each of the databases in parallel and
returns the first response it receives:
</p>
{{code "/doc/progs/timeout2.go" `/func Query/` `/STOP/`}}
<p>
In this example, the closure does a non-blocking send, which it achieves by
using the send operation in <code>select</code> statement with a
<code>default</code> case. If the send cannot go through immediately the
default case will be selected. Making the send non-blocking guarantees that
none of the goroutines launched in the loop will hang around. However, if the
result arrives before the main function has made it to the receive, the send
could fail since no one is ready.
</p>
<p>
This problem is a textbook example of what is known as a
<a href="https://en.wikipedia.org/wiki/Race_condition">race condition</a>, but
the fix is trivial. We just make sure to buffer the channel <code>ch</code> (by
adding the buffer length as the second argument to <a href="/pkg/builtin/#make">make</a>),
guaranteeing that the first send has a place to put the value. This ensures the
send will always succeed, and the first value to arrive will be retrieved
regardless of the order of execution.
</p>
<p>
These two examples demonstrate the simplicity with which Go can express complex
interactions between goroutines.
</p>
doc/articles/defer_panic_recover.html deleted 100644 → 0
View file @ 83da0fd9
<!--{
"Title": "Defer, Panic, and Recover",
"Template": true
}-->
<p>
Go has the usual mechanisms for control flow: if, for, switch, goto. It also
has the go statement to run code in a separate goroutine. Here I'd like to
discuss some of the less common ones: defer, panic, and recover.
</p>
<p>
A <b>defer statement</b> pushes a function call onto a list. The list of saved
calls is executed after the surrounding function returns. Defer is commonly
used to simplify functions that perform various clean-up actions.
</p>
<p>
For example, let's look at a function that opens two files and copies the
contents of one file to the other:
</p>
{{code "/doc/progs/defer.go" `/func CopyFile/` `/STOP/`}}
<p>
This works, but there is a bug. If the call to os.Create fails, the
function will return without closing the source file. This can be easily
remedied by putting a call to src.Close before the second return statement,
but if the function were more complex the problem might not be so easily
noticed and resolved. By introducing defer statements we can ensure that the
files are always closed:
</p>
{{code "/doc/progs/defer2.go" `/func CopyFile/` `/STOP/`}}
<p>
Defer statements allow us to think about closing each file right after opening
it, guaranteeing that, regardless of the number of return statements in the
function, the files <i>will</i> be closed.
</p>
<p>
The behavior of defer statements is straightforward and predictable. There are
three simple rules:
</p>
<p>
1. <i>A deferred function's arguments are evaluated when the defer statement is
evaluated.</i>
</p>
<p>
In this example, the expression "i" is evaluated when the Println call is
deferred. The deferred call will print "0" after the function returns.
</p>
{{code "/doc/progs/defer.go" `/func a/` `/STOP/`}}
<p>
2. <i>Deferred function calls are executed in Last In First Out order
</i>after<i> the surrounding function returns.</i>
</p>
<p>
This function prints "3210":
</p>
{{code "/doc/progs/defer.go" `/func b/` `/STOP/`}}
<p>
3. <i>Deferred functions may read and assign to the returning function's named
return values.</i>
</p>
<p>
In this example, a deferred function increments the return value i <i>after</i>
the surrounding function returns. Thus, this function returns 2:
</p>
{{code "/doc/progs/defer.go" `/func c/` `/STOP/`}}
<p>
This is convenient for modifying the error return value of a function; we will
see an example of this shortly.
</p>
<p>
<b>Panic</b> is a built-in function that stops the ordinary flow of control and
begins <i>panicking</i>. When the function F calls panic, execution of F stops,
any deferred functions in F are executed normally, and then F returns to its
caller. To the caller, F then behaves like a call to panic. The process
continues up the stack until all functions in the current goroutine have
returned, at which point the program crashes. Panics can be initiated by
invoking panic directly. They can also be caused by runtime errors, such as
out-of-bounds array accesses.
</p>
<p>
<b>Recover</b> is a built-in function that regains control of a panicking
goroutine. Recover is only useful inside deferred functions. During normal
execution, a call to recover will return nil and have no other effect. If the
current goroutine is panicking, a call to recover will capture the value given
to panic and resume normal execution.
</p>
<p>
Here's an example program that demonstrates the mechanics of panic and defer:
</p>
{{code "/doc/progs/defer2.go" `/package main/` `/STOP/`}}
<p>
The function g takes the int i, and panics if i is greater than 3, or else it
calls itself with the argument i+1. The function f defers a function that calls
recover and prints the recovered value (if it is non-nil). Try to picture what
the output of this program might be before reading on.
</p>
<p>
The program will output:
</p>
<pre>Calling g.
Printing in g 0
Printing in g 1
Printing in g 2
Printing in g 3
Panicking!
Defer in g 3
Defer in g 2
Defer in g 1
Defer in g 0
Recovered in f 4
Returned normally from f.</pre>
<p>
If we remove the deferred function from f the panic is not recovered and
reaches the top of the goroutine's call stack, terminating the program. This
modified program will output:
</p>
<pre>Calling g.
Printing in g 0
Printing in g 1
Printing in g 2
Printing in g 3
Panicking!
Defer in g 3
Defer in g 2
Defer in g 1
Defer in g 0
panic: 4
panic PC=0x2a9cd8
[stack trace omitted]</pre>
<p>
For a real-world example of <b>panic</b> and <b>recover</b>, see the
<a href="/pkg/encoding/json/">json package</a> from the Go standard library.
It decodes JSON-encoded data with a set of recursive functions.
When malformed JSON is encountered, the parser calls panic to unwind the
stack to the top-level function call, which recovers from the panic and returns
an appropriate error value (see the 'error' and 'unmarshal' methods of
the decodeState type in
<a href="/src/pkg/encoding/json/decode.go">decode.go</a>).
</p>
<p>
The convention in the Go libraries is that even when a package uses panic
internally, its external API still presents explicit error return values.
</p>
<p>
Other uses of <b>defer</b> (beyond the file.Close example given earlier)
include releasing a mutex:
</p>
<pre>mu.Lock()
defer mu.Unlock()</pre>
<p>
printing a footer:
</p>
<pre>printHeader()
defer printFooter()</pre>
<p>
and more.
</p>
<p>
In summary, the defer statement (with or without panic and recover) provides an
unusual and powerful mechanism for control flow. It can be used to model a
number of features implemented by special-purpose structures in other
programming languages. Try it out.
</p>
doc/articles/error_handling.html deleted 100644 → 0
View file @ 83da0fd9
<!--{
"Title": "Error Handling and Go",
"Template": true
}-->
<p>
If you have written any Go code you have probably encountered the built-in
<code>error</code> type. Go code uses <code>error</code> values to
indicate an abnormal state. For example, the <code>os.Open</code> function
returns a non-nil <code>error</code> value when it fails to open a file.
</p>
{{code "/doc/progs/error.go" `/func Open/`}}
<p>
The following code uses <code>os.Open</code> to open a file. If an error
occurs it calls <code>log.Fatal</code> to print the error message and stop.
</p>
{{code "/doc/progs/error.go" `/func openFile/` `/STOP/`}}
<p>
You can get a lot done in Go knowing just this about the <code>error</code>
type, but in this article we'll take a closer look at <code>error</code> and
discuss some good practices for error handling in Go.
</p>
<p>
<b>The error type</b>
</p>
<p>
The <code>error</code> type is an interface type. An <code>error</code>
variable represents any value that can describe itself as a string. Here is the
interface's declaration:
</p>
<pre>type error interface {
Error() string
}</pre>
<p>
The <code>error</code> type, as with all built in types, is
<a href="/doc/go_spec.html#Predeclared_identifiers">predeclared</a> in the
<a href="/doc/go_spec.html#Blocks">universe block</a>.
</p>
<p>
The most commonly-used <code>error</code> implementation is the
<a href="/pkg/errors/">errors</a> package's unexported <code>errorString</code> type.
</p>
{{code "/doc/progs/error.go" `/errorString/` `/STOP/`}}
<p>
You can construct one of these values with the <code>errors.New</code>
function. It takes a string that it converts to an <code>errors.errorString</code>
and returns as an <code>error</code> value.
</p>
{{code "/doc/progs/error.go" `/New/` `/STOP/`}}
<p>
Here's how you might use <code>errors.New</code>:
</p>
{{code "/doc/progs/error.go" `/func Sqrt/` `/STOP/`}}
<p>
A caller passing a negative argument to <code>Sqrt</code> receives a non-nil
<code>error</code> value (whose concrete representation is an
<code>errors.errorString</code> value). The caller can access the error string
("math: square root of...") by calling the <code>error</code>'s
<code>Error</code> method, or by just printing it:
</p>
{{code "/doc/progs/error.go" `/func printErr/` `/STOP/`}}
<p>
The <a href="/pkg/fmt/">fmt</a> package formats an <code>error</code> value
by calling its <code>Error() string</code> method.
</p>
<p>
It is the error implementation's responsibility to summarize the context.
The error returned by <code>os.Open</code> formats as "open /etc/passwd:
permission denied," not just "permission denied." The error returned by our
<code>Sqrt</code> is missing information about the invalid argument.
</p>
<p>
To add that information, a useful function is the <code>fmt</code> package's
<code>Errorf</code>. It formats a string according to <code>Printf</code>'s
rules and returns it as an <code>error</code> created by
<code>errors.New</code>.
</p>
{{code "/doc/progs/error.go" `/fmtError/` `/STOP/`}}
<p>
In many cases <code>fmt.Errorf</code> is good enough, but since
<code>error</code> is an interface, you can use arbitrary data structures as
error values, to allow callers to inspect the details of the error.
</p>
<p>
For instance, our hypothetical callers might want to recover the invalid
argument passed to <code>Sqrt</code>. We can enable that by defining a new
error implementation instead of using <code>errors.errorString</code>:
</p>
{{code "/doc/progs/error.go" `/type NegativeSqrtError/` `/STOP/`}}
<p>
A sophisticated caller can then use a
<a href="/doc/go_spec.html#Type_assertions">type assertion</a> to check for a
<code>NegativeSqrtError</code> and handle it specially, while callers that just
pass the error to <code>fmt.Println</code> or <code>log.Fatal</code> will see
no change in behavior.
</p>
<p>
As another example, the <a href="/pkg/encoding/json/">json</a> package specifies a
<code>SyntaxError</code> type that the <code>json.Decode</code> function
returns when it encounters a syntax error parsing a JSON blob.
</p>
{{code "/doc/progs/error.go" `/type SyntaxError/` `/STOP/`}}
<p>
The <code>Offset</code> field isn't even shown in the default formatting of the
error, but callers can use it to add file and line information to their error
messages:
</p>
{{code "/doc/progs/error.go" `/func decodeError/` `/STOP/`}}
<p>
(This is a slightly simplified version of some
<a href="http://golang.org/s/camjsondecode">actual code</a>
from the <a href="http://camlistore.org">Camlistore</a> project.)
</p>
<p>
The <code>error</code> interface requires only a <code>Error</code> method;
specific error implementations might have additional methods. For instance, the
<a href="/pkg/net/">net</a> package returns errors of type
<code>error</code>, following the usual convention, but some of the error
implementations have additional methods defined by the <code>net.Error</code>
interface:
</p>
<pre>package net
type Error interface {
error
Timeout() bool // Is the error a timeout?
Temporary() bool // Is the error temporary?
}</pre>
<p>
Client code can test for a <code>net.Error</code> with a type assertion and
then distinguish transient network errors from permanent ones. For instance, a
web crawler might sleep and retry when it encounters a temporary error and give
up otherwise.
</p>
{{code "/doc/progs/error.go" `/func netError/` `/STOP/`}}
<p>
<b>Simplifying repetitive error handling</b>
</p>
<p>
In Go, error handling is important. The language's design and conventions
encourage you to explicitly check for errors where they occur (as distinct from
the convention in other languages of throwing exceptions and sometimes catching
them). In some cases this makes Go code verbose, but fortunately there are some
techniques you can use to minimize repetitive error handling.
</p>
<p>
Consider an <a href="http://code.google.com/appengine/docs/go/">App Engine</a>
application with an HTTP handler that retrieves a record from the datastore and
formats it with a template.
</p>
{{code "/doc/progs/error2.go" `/func init/` `/STOP/`}}
<p>
This function handles errors returned by the <code>datastore.Get</code>
function and <code>viewTemplate</code>'s <code>Execute</code> method. In both
cases, it presents a simple error message to the user with the HTTP status code
500 ("Internal Server Error"). This looks like a manageable amount of code, but
add some more HTTP handlers and you quickly end up with many copies of
identical error handling code.
</p>
<p>
To reduce the repetition we can define our own HTTP <code>appHandler</code>
type that includes an <code>error</code> return value:
</p>
{{code "/doc/progs/error3.go" `/type appHandler/`}}
<p>
Then we can change our <code>viewRecord</code> function to return errors:
</p>
{{code "/doc/progs/error3.go" `/func viewRecord/` `/STOP/`}}
<p>
This is simpler than the original version, but the <a
href="/pkg/net/http/">http</a> package doesn't understand functions that return
<code>error</code>.
To fix this we can implement the <code>http.Handler</code> interface's
<code>ServeHTTP</code> method on <code>appHandler</code>:
</p>
{{code "/doc/progs/error3.go" `/ServeHTTP/` `/STOP/`}}
<p>
The <code>ServeHTTP</code> method calls the <code>appHandler</code> function
and displays the returned error (if any) to the user. Notice that the method's
receiver, <code>fn</code>, is a function. (Go can do that!) The method invokes
the function by calling the receiver in the expression <code>fn(w, r)</code>.
</p>
<p>
Now when registering <code>viewRecord</code> with the http package we use the
<code>Handle</code> function (instead of <code>HandleFunc</code>) as
<code>appHandler</code> is an <code>http.Handler</code> (not an
<code>http.HandlerFunc</code>).
</p>
{{code "/doc/progs/error3.go" `/func init/` `/STOP/`}}
<p>
With this basic error handling infrastructure in place, we can make it more
user friendly. Rather than just displaying the error string, it would be better
to give the user a simple error message with an appropriate HTTP status code,
while logging the full error to the App Engine developer console for debugging
purposes.
</p>
<p>
To do this we create an <code>appError</code> struct containing an
<code>error</code> and some other fields:
</p>
{{code "/doc/progs/error4.go" `/type appError/` `/STOP/`}}
<p>
Next we modify the appHandler type to return <code>*appError</code> values:
</p>
{{code "/doc/progs/error4.go" `/type appHandler/`}}
<p>
(It's usually a mistake to pass back the concrete type of an error rather than
<code>error</code>,
for reasons discussed in <a href="/doc/go_faq.html#nil_error">the Go FAQ</a>,
but it's the right thing to do here because <code>ServeHTTP</code> is the only
place that sees the value and uses its contents.)
</p>
<p>
And make <code>appHandler</code>'s <code>ServeHTTP</code> method display the
<code>appError</code>'s <code>Message</code> to the user with the correct HTTP
status <code>Code</code> and log the full <code>Error</code> to the developer
console:
</p>
{{code "/doc/progs/error4.go" `/ServeHTTP/` `/STOP/`}}
<p>
Finally, we update <code>viewRecord</code> to the new function signature and
have it return more context when it encounters an error:
</p>
{{code "/doc/progs/error4.go" `/func viewRecord/` `/STOP/`}}
<p>
This version of <code>viewRecord</code> is the same length as the original, but
now each of those lines has specific meaning and we are providing a friendlier
user experience.
</p>
<p>
It doesn't end there; we can further improve the error handling in our
application. Some ideas:
</p>
<ul>
<li>give the error handler a pretty HTML template,
<li>make debugging easier by writing the stack trace to the HTTP response when
the user is an administrator,
<li>write a constructor function for <code>appError</code> that stores the
stack trace for easier debugging,
<li>recover from panics inside the <code>appHandler</code>, logging the error
to the console as "Critical," while telling the user "a serious error
has occurred." This is a nice touch to avoid exposing the user to inscrutable
error messages caused by programming errors.
See the <a href="defer_panic_recover.html">Defer, Panic, and Recover</a>
article for more details.
</ul>
<p>
<b>Conclusion</b>
</p>
<p>
Proper error handling is an essential requirement of good software. By
employing the techniques described in this post you should be able to write
more reliable and succinct Go code.
</p>
doc/articles/gobs_of_data.html deleted 100644 → 0
View file @ 83da0fd9
<!--{
"Title": "Gobs of data",
"Template": true
}-->
<p>
To transmit a data structure across a network or to store it in a file, it must
be encoded and then decoded again. There are many encodings available, of
course: <a href="http://www.json.org/">JSON</a>,
<a href="http://www.w3.org/XML/">XML</a>, Google's
<a href="http://code.google.com/p/protobuf">protocol buffers</a>, and more.
And now there's another, provided by Go's <a href="/pkg/encoding/gob/">gob</a>
package.
</p>
<p>
Why define a new encoding? It's a lot of work and redundant at that. Why not
just use one of the existing formats? Well, for one thing, we do! Go has
<a href="/pkg/">packages</a> supporting all the encodings just mentioned (the
<a href="http://code.google.com/p/goprotobuf">protocol buffer package</a> is in
a separate repository but it's one of the most frequently downloaded). And for
many purposes, including communicating with tools and systems written in other
languages, they're the right choice.
</p>
<p>
But for a Go-specific environment, such as communicating between two servers
written in Go, there's an opportunity to build something much easier to use and
possibly more efficient.
</p>
<p>
Gobs work with the language in a way that an externally-defined,
language-independent encoding cannot. At the same time, there are lessons to be
learned from the existing systems.
</p>
<p>
<b>Goals</b>
</p>
<p>
The gob package was designed with a number of goals in mind.
</p>
<p>
First, and most obvious, it had to be very easy to use. First, because Go has
reflection, there is no need for a separate interface definition language or
"protocol compiler". The data structure itself is all the package should need
to figure out how to encode and decode it. On the other hand, this approach
means that gobs will never work as well with other languages, but that's OK:
gobs are unashamedly Go-centric.
</p>
<p>
Efficiency is also important. Textual representations, exemplified by XML and
JSON, are too slow to put at the center of an efficient communications network.
A binary encoding is necessary.
</p>
<p>
Gob streams must be self-describing. Each gob stream, read from the beginning,
contains sufficient information that the entire stream can be parsed by an
agent that knows nothing a priori about its contents. This property means that
you will always be able to decode a gob stream stored in a file, even long
after you've forgotten what data it represents.
</p>
<p>
There were also some things to learn from our experiences with Google protocol
buffers.
</p>
<p>
<b>Protocol buffer misfeatures</b>
</p>
<p>
Protocol buffers had a major effect on the design of gobs, but have three
features that were deliberately avoided. (Leaving aside the property that
protocol buffers aren't self-describing: if you don't know the data definition
used to encode a protocol buffer, you might not be able to parse it.)
</p>
<p>
First, protocol buffers only work on the data type we call a struct in Go. You
can't encode an integer or array at the top level, only a struct with fields
inside it. That seems a pointless restriction, at least in Go. If all you want
to send is an array of integers, why should you have to put it into a
struct first?
</p>
<p>
Next, a protocol buffer definition may specify that fields <code>T.x</code> and
<code>T.y</code> are required to be present whenever a value of type
<code>T</code> is encoded or decoded. Although such required fields may seem
like a good idea, they are costly to implement because the codec must maintain a
separate data structure while encoding and decoding, to be able to report when
required fields are missing. They're also a maintenance problem. Over time, one
may want to modify the data definition to remove a required field, but that may
cause existing clients of the data to crash. It's better not to have them in the
encoding at all. (Protocol buffers also have optional fields. But if we don't
have required fields, all fields are optional and that's that. There will be
more to say about optional fields a little later.)
</p>
<p>
The third protocol buffer misfeature is default values. If a protocol buffer
omits the value for a "defaulted" field, then the decoded structure behaves as
if the field were set to that value. This idea works nicely when you have
getter and setter methods to control access to the field, but is harder to
handle cleanly when the container is just a plain idiomatic struct. Required
fields are also tricky to implement: where does one define the default values,
what types do they have (is text UTF-8? uninterpreted bytes? how many bits in a
float?) and despite the apparent simplicity, there were a number of
complications in their design and implementation for protocol buffers. We
decided to leave them out of gobs and fall back to Go's trivial but effective
defaulting rule: unless you set something otherwise, it has the "zero value"
for that type - and it doesn't need to be transmitted.
</p>
<p>
So gobs end up looking like a sort of generalized, simplified protocol buffer.
How do they work?
</p>
<p>
<b>Values</b>
</p>
<p>
The encoded gob data isn't about <code>int8</code>s and <code>uint16</code>s.
Instead, somewhat analogous to constants in Go, its integer values are abstract,
sizeless numbers, either signed or unsigned. When you encode an
<code>int8</code>, its value is transmitted as an unsized, variable-length
integer. When you encode an <code>int64</code>, its value is also transmitted as
an unsized, variable-length integer. (Signed and unsigned are treated
distinctly, but the same unsized-ness applies to unsigned values too.) If both
have the value 7, the bits sent on the wire will be identical. When the receiver
decodes that value, it puts it into the receiver's variable, which may be of
arbitrary integer type. Thus an encoder may send a 7 that came from an
<code>int8</code>, but the receiver may store it in an <code>int64</code>. This
is fine: the value is an integer and as a long as it fits, everything works. (If
it doesn't fit, an error results.) This decoupling from the size of the variable
gives some flexibility to the encoding: we can expand the type of the integer
variable as the software evolves, but still be able to decode old data.
</p>
<p>
This flexibility also applies to pointers. Before transmission, all pointers are
flattened. Values of type <code>int8</code>, <code>*int8</code>,
<code>**int8</code>, <code>****int8</code>, etc. are all transmitted as an
integer value, which may then be stored in <code>int</code> of any size, or
<code>*int</code>, or <code>******int</code>, etc. Again, this allows for
flexibility.
</p>
<p>
Flexibility also happens because, when decoding a struct, only those fields
that are sent by the encoder are stored in the destination. Given the value
</p>
{{code "/doc/progs/gobs1.go" `/type T/` `/STOP/`}}
<p>
the encoding of <code>t</code> sends only the 7 and 8. Because it's zero, the
value of <code>Y</code> isn't even sent; there's no need to send a zero value.
</p>
<p>
The receiver could instead decode the value into this structure:
</p>
{{code "/doc/progs/gobs1.go" `/type U/` `/STOP/`}}
<p>
and acquire a value of <code>u</code> with only <code>X</code> set (to the
address of an <code>int8</code> variable set to 7); the <code>Z</code> field is
ignored - where would you put it? When decoding structs, fields are matched by
name and compatible type, and only fields that exist in both are affected. This
simple approach finesses the "optional field" problem: as the type
<code>T</code> evolves by adding fields, out of date receivers will still
function with the part of the type they recognize. Thus gobs provide the
important result of optional fields - extensibility - without any additional
mechanism or notation.
</p>
<p>
From integers we can build all the other types: bytes, strings, arrays, slices,
maps, even floats. Floating-point values are represented by their IEEE 754
floating-point bit pattern, stored as an integer, which works fine as long as
you know their type, which we always do. By the way, that integer is sent in
byte-reversed order because common values of floating-point numbers, such as
small integers, have a lot of zeros at the low end that we can avoid
transmitting.
</p>
<p>
One nice feature of gobs that Go makes possible is that they allow you to define
your own encoding by having your type satisfy the
<a href="/pkg/encoding/gob/#GobEncoder">GobEncoder</a> and
<a href="/pkg/encoding/gob/#GobDecoder">GobDecoder</a> interfaces, in a manner
analogous to the <a href="/pkg/encoding/json/">JSON</a> package's
<a href="/pkg/encoding/json/#Marshaler">Marshaler</a> and
<a href="/pkg/encoding/json/#Unmarshaler">Unmarshaler</a> and also to the
<a href="/pkg/fmt/#Stringer">Stringer</a> interface from
<a href="/pkg/fmt/">package fmt</a>. This facility makes it possible to
represent special features, enforce constraints, or hide secrets when you
transmit data. See the <a href="/pkg/encoding/gob/">documentation</a> for
details.
</p>
<p>
<b>Types on the wire</b>
</p>
<p>
The first time you send a given type, the gob package includes in the data
stream a description of that type. In fact, what happens is that the encoder is
used to encode, in the standard gob encoding format, an internal struct that
describes the type and gives it a unique number. (Basic types, plus the layout
of the type description structure, are predefined by the software for
bootstrapping.) After the type is described, it can be referenced by its type
number.
</p>
<p>
Thus when we send our first type <code>T</code>, the gob encoder sends a
description of <code>T</code> and tags it with a type number, say 127. All
values, including the first, are then prefixed by that number, so a stream of
<code>T</code> values looks like:
</p>
<pre>
("define type id" 127, definition of type T)(127, T value)(127, T value), ...
</pre>
<p>
These type numbers make it possible to describe recursive types and send values
of those types. Thus gobs can encode types such as trees:
</p>
{{code "/doc/progs/gobs1.go" `/type Node/` `/STOP/`}}
<p>
(It's an exercise for the reader to discover how the zero-defaulting rule makes
this work, even though gobs don't represent pointers.)
</p>
<p>
With the type information, a gob stream is fully self-describing except for the
set of bootstrap types, which is a well-defined starting point.
</p>
<p>
<b>Compiling a machine</b>
</p>
<p>
The first time you encode a value of a given type, the gob package builds a
little interpreted machine specific to that data type. It uses reflection on
the type to construct that machine, but once the machine is built it does not
depend on reflection. The machine uses package unsafe and some trickery to
convert the data into the encoded bytes at high speed. It could use reflection
and avoid unsafe, but would be significantly slower. (A similar high-speed
approach is taken by the protocol buffer support for Go, whose design was
influenced by the implementation of gobs.) Subsequent values of the same type
use the already-compiled machine, so they can be encoded right away.
</p>
<p>
Decoding is similar but harder. When you decode a value, the gob package holds
a byte slice representing a value of a given encoder-defined type to decode,
plus a Go value into which to decode it. The gob package builds a machine for
that pair: the gob type sent on the wire crossed with the Go type provided for
decoding. Once that decoding machine is built, though, it's again a
reflectionless engine that uses unsafe methods to get maximum speed.
</p>
<p>
<b>Use</b>
</p>
<p>
There's a lot going on under the hood, but the result is an efficient,
easy-to-use encoding system for transmitting data. Here's a complete example
showing differing encoded and decoded types. Note how easy it is to send and
receive values; all you need to do is present values and variables to the
<a href="/pkg/encoding/gob/">gob package</a> and it does all the work.
</p>
{{code "/doc/progs/gobs2.go" `/package main/` `$`}}
<p>
You can compile and run this example code in the
<a href="http://play.golang.org/p/_-OJV-rwMq">Go Playground</a>.
</p>
<p>
The <a href="/pkg/net/rpc/">rpc package</a> builds on gobs to turn this
encode/decode automation into transport for method calls across the network.
That's a subject for another article.
</p>
<p>
<b>Details</b>
</p>
<p>
The <a href="/pkg/encoding/gob/">gob package documentation</a>, especially the
file <a href="/src/pkg/encoding/gob/doc.go">doc.go</a>, expands on many of the
details described here and includes a full worked example showing how the
encoding represents data. If you are interested in the innards of the gob
implementation, that's a good place to start.
</p>
doc/articles/godoc_documenting_go_code.html deleted 100644 → 0
View file @ 83da0fd9
<!--{
"Title": "Godoc: documenting Go code",
"Template": true
}-->
<p>
The Go project takes documentation seriously. Documentation is a huge part of
making software accessible and maintainable. Of course it must be well-written
and accurate, but it also must be easy to write and to maintain. Ideally, it
should be coupled to the code itself so the documentation evolves along with the
code. The easier it is for programmers to produce good documentation, the better
for everyone.
</p>
<p>
To that end, we have developed the <a href="/cmd/godoc/">godoc</a> documentation
tool. This article describes godoc's approach to documentation, and explains how
you can use our conventions and tools to write good documentation for your own
projects.
</p>
<p>
Godoc parses Go source code - including comments - and produces documentation as
HTML or plain text. The end result is documentation tightly coupled with the
code it documents. For example, through godoc's web interface you can navigate
from a function's <a href="/pkg/strings/#HasPrefix">documentation</a> to its
<a href="/src/pkg/strings/strings.go?#L312">implementation</a> with one click.
</p>
<p>
Godoc is conceptually related to Python's
<a href="http://www.python.org/dev/peps/pep-0257/">Docstring</a> and Java's
<a href="http://www.oracle.com/technetwork/java/javase/documentation/index-jsp-135444.html">Javadoc</a>,
but its design is simpler. The comments read by godoc are not language
constructs (as with Docstring) nor must they have their own machine-readable
syntax (as with Javadoc). Godoc comments are just good comments, the sort you
would want to read even if godoc didn't exist.
</p>
<p>
The convention is simple: to document a type, variable, constant, function, or
even a package, write a regular comment directly preceding its declaration, with
no intervening blank line. Godoc will then present that comment as text
alongside the item it documents. For example, this is the documentation for the
<code>fmt</code> package's <a href="/pkg/fmt/#Fprint"><code>Fprint</code></a>
function:
</p>
{{code "/src/pkg/fmt/print.go" `/Fprint formats using the default/` `/func Fprint/`}}
<p>
Notice this comment is a complete sentence that begins with the name of the
element it describes. This important convention allows us to generate
documentation in a variety of formats, from plain text to HTML to UNIX man
pages, and makes it read better when tools truncate it for brevity, such as when
they extract the first line or sentence.
</p>
<p>
Comments on package declarations should provide general package documentation.
These comments can be short, like the <a href="/pkg/sort/"><code>sort</code></a>
package's brief description:
</p>
{{code "/src/pkg/sort/sort.go" `/Package sort provides/` `/package sort/`}}
<p>
They can also be detailed like the <a href="/pkg/encoding/gob/"><code>gob</code></a>
package's overview. That package uses another convention for packages
that need large amounts of introductory documentation: the package comment is
placed in its own file, <a href="/src/pkg/encoding/gob/doc.go">doc.go</a>, which
contains only those comments and a package clause.
</p>
<p>
When writing package comments of any size, keep in mind that their first
sentence will appear in godoc's <a href="/pkg/">package list</a>.
</p>
<p>
Comments that are not adjacent to a top-level declaration are omitted from
godoc's output, with one notable exception. Top-level comments that begin with
the word <code>"BUG(who)"</code> are recognized as known bugs, and included in
the "Bugs" section of the package documentation. The "who" part should be the
user name of someone who could provide more information. For example, this is a
known issue from the <a href="/pkg/sync/atomic/#pkg-note-BUG"><code>sync/atomic</code></a> package:
</p>
<pre>
// BUG(rsc): On x86-32, the 64-bit functions use instructions
// unavailable before the Pentium MMX. On both ARM and x86-32, it is the
// caller's responsibility to arrange for 64-bit alignment of 64-bit
// words accessed atomically.
</pre>
<p>
Godoc treats executable commands in the same way. It looks for a comment on
package main, which is sometimes put in a separate file called <code>doc.go</code>.
For example, see the
<a href="/cmd/godoc/">godoc documentation</a> and its corresponding
<a href="/src/cmd/godoc/doc.go">doc.go</a> file.
</p>
<p>
There are a few formatting rules that Godoc uses when converting comments to
HTML:
</p>
<ul>
<li>
Subsequent lines of text are considered part of the same paragraph; you must
leave a blank line to separate paragraphs.
</li>
<li>
Pre-formatted text must be indented relative to the surrounding comment text
(see gob's <a href="/src/pkg/encoding/gob/doc.go">doc.go</a> for an example).
</li>
<li>
URLs will be converted to HTML links; no special markup is necessary.
</li>
</ul>
<p>
Note that none of these rules requires you to do anything out of the ordinary.
</p>
<p>
In fact, the best thing about godoc's minimal approach is how easy it is to use.
As a result, a lot of Go code, including all of the standard library, already
follows the conventions.
</p>
<p>
Your own code can present good documentation just by having comments as
described above. Any Go packages installed inside <code>$GOROOT/src/pkg</code>
and any <code>GOPATH</code> work spaces will already be accessible via godoc's
command-line and HTTP interfaces, and you can specify additional paths for
indexing via the <code>-path</code> flag or just by running <code>"godoc ."</code>
in the source directory. See the <a href="/cmd/godoc/">godoc documentation</a>
for more details.
</p>
<p>
Godoc recognizes example functions written according to the
<a href="/pkg/testing/#pkg-overview"><code>testing</code></a> package's naming
conventions and presents them appropriately.
</p>
doc/articles/gos_declaration_syntax.html deleted 100644 → 0
View file @ 83da0fd9
<!--{
"Title": "Go's Declaration Syntax"
}-->
<p>
Newcomers to Go wonder why the declaration syntax is different from the
tradition established in the C family. In this post we'll compare the
two approaches and explain why Go's declarations look as they do.
</p>
<p>
<b>C syntax</b>
</p>
<p>
First, let's talk about C syntax. C took an unusual and clever approach
to declaration syntax. Instead of describing the types with special
syntax, one writes an expression involving the item being declared, and
states what type that expression will have. Thus
</p>
<pre>
int x;
</pre>
<p>
declares x to be an int: the expression 'x' will have type int. In
general, to figure out how to write the type of a new variable, write an
expression involving that variable that evaluates to a basic type, then
put the basic type on the left and the expression on the right.
</p>
<p>
Thus, the declarations
</p>
<pre>
int *p;
int a[3];
</pre>
<p>
state that p is a pointer to int because '*p' has type int, and that a
is an array of ints because a[3] (ignoring the particular index value,
which is punned to be the size of the array) has type int.
</p>
<p>
What about functions? Originally, C's function declarations wrote the
types of the arguments outside the parens, like this:
</p>
<pre>
int main(argc, argv)
int argc;
char *argv[];
{ /* ... */ }
</pre>
<p>
Again, we see that main is a function because the expression main(argc,
argv) returns an int. In modern notation we'd write
</p>
<pre>
int main(int argc, char *argv[]) { /* ... */ }
</pre>
<p>
but the basic structure is the same.
</p>
<p>
This is a clever syntactic idea that works well for simple types but can
get confusing fast. The famous example is declaring a function pointer.
Follow the rules and you get this:
</p>
<pre>
int (*fp)(int a, int b);
</pre>
<p>
Here, fp is a pointer to a function because if you write the expression
(*fp)(a, b) you'll call a function that returns int. What if one of fp's
arguments is itself a function?
</p>
<pre>
int (*fp)(int (*ff)(int x, int y), int b)
</pre>
<p>
That's starting to get hard to read.
</p>
<p>
Of course, we can leave out the name of the parameters when we declare a
function, so main can be declared
</p>
<pre>
int main(int, char *[])
</pre>
<p>
Recall that argv is declared like this,
</p>
<pre>
char *argv[]
</pre>
<p>
so you drop the name from the <em>middle</em> of its declaration to construct
its type. It's not obvious, though, that you declare something of type
char *[] by putting its name in the middle.
</p>
<p>
And look what happens to fp's declaration if you don't name the
parameters:
</p>
<pre>
int (*fp)(int (*)(int, int), int)
</pre>
<p>
Not only is it not obvious where to put the name inside
</p>
<pre>
int (*)(int, int)
</pre>
<p>
it's not exactly clear that it's a function pointer declaration at all.
And what if the return type is a function pointer?
</p>
<pre>
int (*(*fp)(int (*)(int, int), int))(int, int)
</pre>
<p>
It's hard even to see that this declaration is about fp.
</p>
<p>
You can construct more elaborate examples but these should illustrate
some of the difficulties that C's declaration syntax can introduce.
</p>
<p>
There's one more point that needs to be made, though. Because type and
declaration syntax are the same, it can be difficult to parse
expressions with types in the middle. This is why, for instance, C casts
always parenthesize the type, as in
</p>
<pre>
(int)M_PI
</pre>
<p>
<b>Go syntax</b>
</p>
<p>
Languages outside the C family usually use a distinct type syntax in
declarations. Although it's a separate point, the name usually comes
first, often followed by a colon. Thus our examples above become
something like (in a fictional but illustrative language)
</p>
<pre>
x: int
p: pointer to int
a: array[3] of int
</pre>
<p>
These declarations are clear, if verbose - you just read them left to
right. Go takes its cue from here, but in the interests of brevity it
drops the colon and removes some of the keywords:
</p>
<pre>
x int
p *int
a [3]int
</pre>
<p>
There is no direct correspondence between the look of [3]int and how to
use a in an expression. (We'll come back to pointers in the next
section.) You gain clarity at the cost of a separate syntax.
</p>
<p>
Now consider functions. Let's transcribe the declaration for main, even
though the main function in Go takes no arguments:
</p>
<pre>
func main(argc int, argv *[]byte) int
</pre>
<p>
Superficially that's not much different from C, but it reads well from
left to right:
</p>
<p>
<em>function main takes an int and a pointer to a slice of bytes and returns an int.</em>
</p>
<p>
Drop the parameter names and it's just as clear - they're always first
so there's no confusion.
</p>
<pre>
func main(int, *[]byte) int
</pre>
<p>
One value of this left-to-right style is how well it works as the types
become more complex. Here's a declaration of a function variable
(analogous to a function pointer in C):
</p>
<pre>
f func(func(int,int) int, int) int
</pre>
<p>
Or if f returns a function:
</p>
<pre>
f func(func(int,int) int, int) func(int, int) int
</pre>
<p>
It still reads clearly, from left to right, and it's always obvious
which name is being declared - the name comes first.
</p>
<p>
The distinction between type and expression syntax makes it easy to
write and invoke closures in Go:
</p>
<pre>
sum := func(a, b int) int { return a+b } (3, 4)
</pre>
<p>
<b>Pointers</b>
</p>
<p>
Pointers are the exception that proves the rule. Notice that in arrays
and slices, for instance, Go's type syntax puts the brackets on the left
of the type but the expression syntax puts them on the right of the
expression:
</p>
<pre>
var a []int
x = a[1]
</pre>
<p>
For familiarity, Go's pointers use the * notation from C, but we could
not bring ourselves to make a similar reversal for pointer types. Thus
pointers work like this
</p>
<pre>
var p *int
x = *p
</pre>
<p>
We couldn't say
</p>
<pre>
var p *int
x = p*
</pre>
<p>
because that postfix * would conflate with multiplication. We could have
used the Pascal ^, for example:
</p>
<pre>
var p ^int
x = p^
</pre>
<p>
and perhaps we should have (and chosen another operator for xor),
because the prefix asterisk on both types and expressions complicates
things in a number of ways. For instance, although one can write
</p>
<pre>
[]int("hi")
</pre>
<p>
as a conversion, one must parenthesize the type if it starts with a *:
</p>
<pre>
(*int)(nil)
</pre>
<p>
Had we been willing to give up * as pointer syntax, those parentheses
would be unnecessary.
</p>
<p>
So Go's pointer syntax is tied to the familiar C form, but those ties
mean that we cannot break completely from using parentheses to
disambiguate types and expressions in the grammar.
</p>
<p>
Overall, though, we believe Go's type syntax is easier to understand
than C's, especially when things get complicated.
</p>
<p>
<b>Notes</b>
</p>
<p>
Go's declarations read left to right. It's been pointed out that C's
read in a spiral! See <a href="http://c-faq.com/decl/spiral.anderson.html">
The "Clockwise/Spiral Rule"</a> by David Anderson.
</p>
doc/articles/image-20.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-20.png

93.1 KB

doc/articles/image-2a.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-2a.png

3.54 KB

doc/articles/image-2b.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-2b.png

93.2 KB

doc/articles/image-2c.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-2c.png

59.1 KB

doc/articles/image-2d.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-2d.png

66.7 KB

doc/articles/image-2e.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-2e.png

94.5 KB

doc/articles/image-2f.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-2f.png

61.2 KB

doc/articles/image-package-01.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-package-01.png

1.36 KB

doc/articles/image-package-02.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-package-02.png

1.46 KB

doc/articles/image-package-03.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-package-03.png

1.44 KB

doc/articles/image-package-04.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-package-04.png

1.59 KB

doc/articles/image-package-05.png deleted 100644 → 0
View file @ 83da0fd9
doc/articles/image-package-05.png

1.58 KB

doc/articles/image_draw.html deleted 100644 → 0
View file @ 83da0fd9
<!--{
"Title": "The Go image/draw package",
"Template": true
}-->
<p>
<a href="/pkg/image/draw/">Package image/draw</a> defines
only one operation: drawing a source image onto a destination
image, through an optional mask image. This one operation is
surprisingly versatile and can perform a number of common image
manipulation tasks elegantly and efficiently.
</p>
<p>
Composition is performed pixel by pixel in the style of the Plan 9
graphics library and the X Render extension. The model is based on
the classic "Compositing Digital Images" paper by Porter and Duff,
with an additional mask parameter: <code>dst = (src IN mask) OP dst</code>.
For a fully opaque mask, this reduces to the original Porter-Duff
formula: <code>dst = src OP dst</code>. In Go, a nil mask image is equivalent
to an infinitely sized, fully opaque mask image.
</p>
<p>
The Porter-Duff paper presented
<a href="http://www.w3.org/TR/SVGCompositing/examples/compop-porterduff-examples.png">12 different composition operators</a>,
but with an explicit mask, only 2 of these are needed in practice:
source-over-destination and source. In Go, these operators are
represented by the <code>Over</code> and <code>Src</code> constants. The <code>Over</code> operator
performs the natural layering of a source image over a destination
image: the change to the destination image is smaller where the
source (after masking) is more transparent (that is, has lower
alpha). The <code>Src</code> operator merely copies the source (after masking)
with no regard for the destination image's original content. For
fully opaque source and mask images, the two operators produce the
same output, but the <code>Src</code> operator is usually faster.
</p>
<p><b>Geometric Alignment</b></p>
<p>
Composition requires associating destination pixels with source and
mask pixels. Obviously, this requires destination, source and mask
images, and a composition operator, but it also requires specifying
what rectangle of each image to use. Not every drawing should write
to the entire destination: when updating an animating image, it is
more efficient to only draw the parts of the image that have
changed. Not every drawing should read from the entire source: when
using a sprite that combines many small images into one large one,
only a part of the image is needed. Not every drawing should read
from the entire mask: a mask image that collects a font's glyphs is
similar to a sprite. Thus, drawing also needs to know three
rectangles, one for each image. Since each rectangle has the same
width and height, it suffices to pass a destination rectangle `r`
and two points <code>sp</code> and <code>mp</code>: the source rectangle is equal to <code>r</code>
translated so that <code>r.Min</code> in the destination image aligns with
<code>sp</code> in the source image, and similarly for <code>mp</code>. The effective
rectangle is also clipped to each image's bounds in their
respective co-ordinate space.
</p>
<p>
<img src="image-20.png">
</p>
<p>
The <a href="/pkg/image/draw/#DrawMask"><code>DrawMask</code></a>
function takes seven arguments, but an explicit mask and mask-point
are usually unnecessary, so the
<a href="/pkg/image/draw/#Draw"><code>Draw</code></a> function takes five:
</p>
<pre>
// Draw calls DrawMask with a nil mask.
func Draw(dst Image, r image.Rectangle, src image.Image, sp image.Point, op Op)
func DrawMask(dst Image, r image.Rectangle, src image.Image, sp image.Point,
mask image.Image, mp image.Point, op Op)
</pre>
<p>
The destination image must be mutable, so the image/draw package
defines a <a href="/pkg/image/draw/#Image"><code>draw.Image</code></a>
interface which has a <code>Set</code> method.
</p>
{{code "../src/pkg/image/draw/draw.go" `/type Image/` `/}/`}}
<p><b>Filling a Rectangle</b></p>
<p>
To fill a rectangle with a solid color, use an <code>image.Uniform</code>
source. The <code>Uniform</code> type re-interprets a <code>Color</code> as a
practically infinite-sized <code>Image</code> of that color. For those
familiar with the design of Plan 9's draw library, there is no need
for an explicit "repeat bit" in Go's slice-based image types; the
concept is subsumed by <code>Uniform</code>.
</p>
{{code "/doc/progs/image_draw.go" `/ZERO/` `/STOP/`}}
<p>
To initialize a new image to all-blue:
</p>
{{code "/doc/progs/image_draw.go" `/BLUE/` `/STOP/`}}
<p>
To reset an image to transparent (or black, if the destination
image's color model cannot represent transparency), use
<code>image.Transparent</code>, which is an <code>image.Uniform</code>:
</p>
{{code "/doc/progs/image_draw.go" `/RESET/` `/STOP/`}}
<p>
<img src="image-2a.png">
</p>
<p><b>Copying an Image</b></p>
<p>
To copy from a rectangle <code>sr</code> in the source image to a rectangle
starting at a point <code>dp</code> in the destination, convert the source
rectangle into the destination image's co-ordinate space:
</p>
{{code "/doc/progs/image_draw.go" `/RECT/` `/STOP/`}}
<p>
Alternatively:
</p>
{{code "/doc/progs/image_draw.go" `/RECT2/` `/STOP/`}}
<p>
To copy the entire source image, use <code>sr = src.Bounds()</code>.
</p>
<p>
<img src="image-2b.png">
</p>
<p><b>Scrolling an Image</b></p>
<p>
Scrolling an image is just copying an image to itself, with
different destination and source rectangles. Overlapping
destination and source images are perfectly valid, just as Go's
built-in copy function can handle overlapping destination and
source slices. To scroll an image m by 20 pixels:
</p>
{{code "/doc/progs/image_draw.go" `/SCROLL/` `/STOP/`}}
<p><img src="image-2c.png"></p>
<p><b>Converting an Image to RGBA</b></p>
<p>
The result of decoding an image format might not be an
<code>image.RGBA</code>: decoding a GIF results in an <code>image.Paletted</code>,
decoding a JPEG results in a <code>ycbcr.YCbCr</code>, and the result of
decoding a PNG depends on the image data. To convert any image to
an <code>image.RGBA</code>:
</p>
{{code "/doc/progs/image_draw.go" `/CONV/` `/STOP/`}}
<p>
<img src="image-2d.png">
</p>
<p><b>Drawing Through a Mask</b></p>
<p>
To draw an image through a circular mask with center <code>p</code> and radius
<code>r</code>:
</p>
{{code "/doc/progs/image_draw.go" `/CIRCLESTRUCT/` `/STOP/`}}
{{code "/doc/progs/image_draw.go" `/CIRCLE2/` `/STOP/`}}
<p>
<img src="image-2e.png">
</p>
<p><b>Drawing Font Glyphs</b></p>
<p>
To draw a font glyph in blue starting from a point <code>p</code>, draw with
an <code>image.Uniform</code> source and an <code>image.Alpha mask</code>. For
simplicity, we aren't performing any sub-pixel positioning or
rendering, or correcting for a font's height above a baseline.
</p>
{{code "/doc/progs/image_draw.go" `/GLYPH/` `/STOP/`}}
<p>
<img src="image-2f.png">
</p>
<p><b>Performance</b></p>
<p>
The image/draw package implementation demonstrates how to provide
an image manipulation function that is both general purpose, yet
efficient for common cases. The <code>DrawMask</code> function takes arguments
of interface types, but immediately makes type assertions that its
arguments are of specific struct types, corresponding to common
operations like drawing one <code>image.RGBA</code> image onto another, or
drawing an <code>image.Alpha</code> mask (such as a font glyph) onto an
<code>image.RGBA</code> image. If a type assertion succeeds, that type
information is used to run a specialized implementation of the
general algorithm. If the assertions fail, the fallback code path
uses the generic <code>At</code> and <code>Set</code> methods. The fast-paths are purely
a performance optimization; the resultant destination image is the
same either way. In practice, only a small number of special cases
are necessary to support typical applications.
</p>
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7