Golang 数据和语义的设计哲学

Sep 2, 2020 20:50 · 6633 words · 14 minute read Golang

设计哲学

“将数据置于栈上能够减小垃圾收集器（GC）的压力。但这样的话就要存储、追踪和维护给定值的多个副本。而指针语义将值置于堆上，虽然给 GC 带来了压力，但是因始终只需存储、追踪和维护一个值而变得高效起来。”—— Bill Kennedy

对于给定类型的数据，如果要在整个软件中保持完整和可读性，一致地使用值或指针语义是至关重要的。如果你在函数调用时而传递值时而传递指针，这样很难维护一个清晰一致的心智模型。代码量和团队规模越大，bug 越多，数据竞争和其他副作用会成为潜在的威胁。

我来讲讲在两者之间权衡的设计哲学。

心智模型

“我们想象有这样一个项目，有一百万行以上代码。这些项目成功的可能性极低，远低于 50%。这也是有争议的说法。” —— Tom Love（Objective C 的创造者）

Tom 还说一盒打印纸可以容纳十万行代码。稍微想想，换做是你可以掌握多少这个盒子中的代码呢？

我相信要一个开发者来维护一万行要求已经不低了。假设每个开发者写一万行，那么一百万行代码的仓库就需要有一百个开发者组成的团队来维护，也就是说一百个人要协调、分组、跟踪和沟通。现在看看你十个人都不到的团队，规模小的多了，干得咋样？按每人一万行，你们团队规模和代码库的体量是否相符？

调试

“最难搞的 bug 是你的思维就有问题，所以根本看不到问题。“ —— Brian Kernighan

我不相信调试器，除非你对代码完全失控并且正在浪费精力试图去理解问题。当调试器被滥用时它们就是恶魔，碰到问题第一反应就是调试器时，这就是滥用。

如果在生产环境碰到问题，找谁问去？没错，日志。如果日志在你开发时都没啥用，当生产环境出现 bug 那多半也没啥用。看日志需要对代码心理有数，这样才能阅读代码来找 bug。

可读性

“C 是我所见过在性能与表达之间平衡得最好的。你可以通过简单的编程实现任何你想做的事，而且对机器上将发生什么了如指掌。你可以合理地预测它的运行速度，因为你知道下面将发生什么……” —— Brian Kernighan

我相信 Brian 这句话同样适用于 Go。维持这种“心智模型”就是一切，它驱动着完整性、可读性和简单性。这些都是高质量的软件的基石，使得它的生命周期得以延续。保持给定类型数据的值或者指针语义一致就是实现这一点的重要方法。

面向数据设计

“如果你不理解数据，那你也别想理解问题。因为所有问题与你所使用的数据关系紧密，问题随着数据的变化而变化。当你碰到的问题改变了，你的算法也要跟着变。” —— Bill Kennedy

试想，你在解决一个数据传输的问题。你所写的每个函数接收一些输入的数据并产生一些输出的数据。从这个角度看，你的心智模型就是对数据转换的理解（比如怎样在代码层面上组织和应用它们）。“少即是多”的原则对于解决问题时实现较少的抽象层数，代码量，迭代次数，以及降低复杂性和减少工作量非常重要。这不仅使你的团队更从容，硬件也更高效地执行这些数据转换。

类型（就是生命）

“完整性意味着每次分配、每次内存读写都是精确、一致和高效的。类型系统对于完整性至关重要。” —— William Kennedy

如果数据驱动着你所做的一切，那么代表数据的类型就非常重要了。在我的世界中“类型就是生命”，因为类型为编译器提供了确保数据完整性的能力。类型也驱动并指示代码遵循的语义规则。这是正确地使用值或者指针语义的前提。

数据（就是能力）

“当数据是实际和合理的，方法才是有效的。” —— William Kennedy

Go 开发者不会直面值或指针语义，直到他们需要权衡方法接收值还是指针。这是我经常遇到的一个问题：应该使用值作为参数还是指针？每当我看到这个问题，就知道这个开发者没有理解好这些语义。

方法的目的在于赋予数据某种能力。我总想要聚焦于数据因为数据驱动着程序的功能，数据也驱动你写算法、封装和提升性能表现。

多态

“多态意味着你写了一个特定的程序，但是面对不同的数据它的操作有所不同。” —— Tom Kurtz（BASIC 的发明者）

我爱 Tom 这段话。函数的行为因操作的数据的不同而不同。数据的行为是将函数与可以接受和处理的具体类型解耦的原因。这是数据可以具备能力的核心理由之一。这个观点是设计可以适应变化的系统架构的基石。

原型优先

“除非开发者对软件会被如何使用了如指掌，否则软件很可能会出问题。如果开发者不能知晓和理解应用程序，那么获得尽可能多的测试就相当的重要” —— Brian Kernighan

我知道你们总是专注于理解具体数据和解决问题的算法。采取原型优先的思想，并编写可以上生产的具体实现（如果这样做合理且实际）。也要专注于重构，通过赋予数据能力来解耦功能实现与具体数据。

语义规范

在声明类型时就必须权衡使用值还是指针语义。接收或返回该类型数据的 API 必须遵循为其选择的语义。必须指定什么类型使用什么语义并遵守它，这是实现大型代码库一致性的基本要求。

下面是基本原则：

声明类型的同时必须确定将要使用的语义
函数与方法必须遵循给定类型所选择的语义
避免方法接收方使用与给定类型不同的语义
避免函数接收/返回与给定类型不同的语义
避免更改给定类型的语义

有些例外，最典型的是 unmarshaling，经常使用指针语义，marshaling 也一样。

如何为给定的类型选择语义呢？下面我们将其应用于具体场景：

内置类型

Go 内置类型包括数字、文本和布尔数据。这些类型应该使用值语义进行处理。如非必要不要用指针来共享这些类型的值。

作为例子，看看 strings 包的一些函数声明。

func Replace(s, old, new string, n int) string
func LastIndex(s, sep string) int
func ContainsRune(s string, r rune) bool

所有这些函数在 API 设计时都使用值语义。

引用类型

引用类型包括切片、map、接口、函数和管道。这些类型应当使用值语义因为它们被设计成置于栈上来最小化堆的压力。它们允许每个函数都有自己的值副本，避免造成潜在的分配。这是可能的，因为它们都包含了在调用之间共享底层数据结构的指针。

如非必要不要用指针来共享这些类型的值。将调用栈中的 map 或切片共享给 Unmarshal 可能是个例外。作为示例，看看 net 包声明的这两种类型。

type IP []byte
type IPMask []byte

IP 和 IPMask 都是字节切片。这意味着它们都是引用类型并且还要遵循值语义。下面是名为 Mask 的方法，被声明为 IP 类型的方法，接收一个 IPMask 类型的值。

func (ip IP) Mask(mask IPMask) IP {
    if len(mask) == IPv6len && len(ip) == IPv4len && allFF(mask[:12]) {
        mask = mask[12:]
    }
    if len(mask) == IPv4len && len(ip) == IPv6len && bytesEqual(ip[:12], v4InV6Prefix) {
        ip = ip[12:]
    }
    n := len(ip)
    if n != len(mask) {
        return nil
    }
    out := make(IP, n)
    for i := 0; i < n; i++ {
        out[i] = ip[i] & mask[i]
    }
    return out
}

要注意这个方法是一种转变操作，并使用值语义的样式。IP 类型的值作为接收者，基于传入的 IPMask 值创建出一个新的 IP 值并将副本返回给调用方。该方法遵循了对引用类型使用值语义的原则。

和内置的 append 函数一样。

var data []string
data = append(data, "string")

append 函数用值语义来处理。把一个切片值传入 append 并返回一个新的切片值。

只有 unmarshaling 总是使用指针语义的。

func (ip *IP) UnmarshalText(text []byte) error {
    if len(text) == 0 {
        *ip = nil
        return nil
    }
    s := string(text)
    x := ParseIP(s)
    if x == nil {
        return &ParseError{Type: "IP address", Text: s}
    }
    *ip = x
    return nil
}

UnmarshalText 方法是 encoding.TextUnmarshaler 接口的实现。如果不使用指针语义就无法实现。这样就可以因为共享值通常很安全。除了 unmarshaling 之外对引用类型使用指针语义都要三思。

用户自定义类型

这是最需要权衡的地方，要在声明的时候就想好用什么语义。

如果给你以下类型并要你自行实现 time 包的 API。

type Time struct {
    sec  int64
    nsec int32
    loc  *Location
}

你会使用什么语义？

查看 Time 包中此类型的实现以及工厂函数 Now。

func Now() Time {
    sec, nsec := now()
    return Time{sec + unixToInternal, nsec, Local}
}

工厂函数对类型来说非常重要，因为它告知你所选择的语义。Now 函数清楚地表明使用了值语义。此函数创建了一个 Time 类型的值并将它的副本返回给了调用方。共享 Time 值没必要，它们的生命周期内不需要一直存在于堆上。

看下 Add 方法，这也是个变化操作。

func (t Time) Add(d Duration) Time {
    t.sec += int64(d / 1e9)
    nsec := t.nsec + int32(d%1e9)
    if nsec >= 1e9 {
        t.sec++
        nsec -= 1e9
    } else if nsec < 0 {
        t.sec--
        nsec += 1e9
    }
    t.nsec = nsec
    return t
}

Add 方法使用一个接收值来操作它自己的 Time 值副本，Time 值副本被用来调用这个方法，随后处理自己的副本并返回 Time 值的副本。

以下是一个接受 Time 值的函数。

func div(t Time, d Duration) (qmod2 int, r Duration) {

值语义又双叒被用来接收 Time 类型的值。唯一使用指针语义的 Time API 都是 Unmarshal 相关的函数：

func (t *Time) UnmarshalBinary(data []byte) error {
func (t *Time) GobDecode(data []byte) error {
func (t *Time) UnmarshalJSON(data []byte) error {
func (t *Time) UnmarshalText(data []byte) error {

大多数情况下使用值语义的能力是有限的。通常值传递是不合理的，修改数据要被隔离起来并共享，这时就应该使用指针了。如果你不能保证副本是正确或合理的，那就用指针吧。

看下 os 包 File 类型的工厂函数。

func Open(name string) (file *File, err error) {
    return OpenFile(name, O_RDONLY, 0)
}

Open 函数返回 File 类型的指针。这意味着你应该使用指针语义并总是共享 File 值。将指针语义改成值语义可能会对你的程序造成毁灭性的影响。当与一个函数共享值时，最好不要拷贝这个值的指针来使用，弄不好会空指针。

查看更多 API，将会看到更多使用指针语义的例子。

func (f *File) Chdir() error {
    if f == nil {
        return ErrInvalid
    }
    if e := syscall.Fchdir(f.fd); e != nil {
        return &PathError{"chdir", f.name, e}
    }
    return nil
}

尽管没有处理 File 值 Chdir 还是使用了指针语义。该方法必须遵循该类型的语义约定。

func epipecheck(file *File, e error) {
    if e == syscall.EPIPE {
        if atomic.AddInt32(&file.nepipe, 1) >= 10 {
            sigpipe()
        }
    } else {
        atomic.StoreInt32(&file.nepipe, 0)
    }
}

这个名为 epipecheck 的方法也使用指针语义来接收 File 值。再次注意下，对于 File 值一致使用指针语义。

结论

我在审查代码时，会寻找值或者指针语义是否使用一致。它可以帮助你保证代码的一致性和可预测性，还使每个人能保持清晰和一致的心智模型。随着代码量和团队规模变得越来越大，一致性使用将会越来越重要。

原文

Design Philosophies

“Value semantics keep values on the stack, which reduces pressure on the Garbage Collector (GC). However, value semantics require various copies of any given value to be stored, tracked and maintained. Pointer semantics place values on the heap, which can put pressure on the GC. However, pointer semantics are efficient because only one value needs to be stored, tracked and maintained.” - Bill Kennedy

A consistent use of value/pointer semantics, for a given type of data, is critical if you want to maintain integrity and readability throughout your software. Why? Because, if you are changing the semantics for a piece of data as it is passed between functions, you are making it difficult to maintain a clear and consistent mental model of the code. The larger the code base and the team becomes, the more bugs, data races and side effects will creep unseen into the code base.

I want to start with a set of design philosophies that will drive the guidelines for choosing one semantic over the other.

Mental Models

“Let’s imagine a project that’s going to end up with a million lines of code or more. The probability of those projects being successful in the United States these days is very low - well under 50%. That’s debatable”. - Tom Love (inventor of Objective C)

Tom has also mentioned that a box of copy paper can hold 100k lines of code. Take a second to let that sink in. For what percentage of the code in that box could you maintain a mental model of?

I believe asking a single developer to maintain a mental model of more than one ream of paper (~10k lines of code) is already asking quite a bit. But, if we go ahead assume ~10k lines of code per developer, it would take a team of 100 developers to maintain on a code base that hits a million lines of code. That is 100 people that need to be coordinated, grouped, tracked and in constant communication. Now look at your current team of 1 to 10 devs. How well are you doing with this at a much smaller scale? At 10k lines of code per developer, is the size of your code base in line with the size of the team?

Debugging

“The hardest bugs are those where your mental model of the situation is just wrong, so you can’t see the problem at all” - Brian Kernighan

I don’t believe in using a debugger until you have lost your mental model of the code and are now wasting effort trying to understand the problem. Debuggers are evil when they are abused, and you know that you are abusing a debugger when it becomes the first reaction to any observable bug.

If you have a problem in production, what are you going to ask for? That’s right, the logs. If the logs are not working for you during development, they are certainly not going to work for you when that production bug hits. Logs require having a mental model of the code base, such that you can read code to find bugs.

Readability

“C is the best balance I’ve ever seen between power and expressiveness. You can do almost anything you want to do by programming fairly straightforwardly and you will have a very good mental model of what’s going to happen on the machine; you can predict reasonably well how quickly it’s going to run, you understand what’s going on ….” - Brian Kernighan

I believe this quote by Brian applies just as well to Go. Maintaining this “mental model” is everything. It drives integrity, readability and simplicity. These are the cornerstones of well-written software that allow it to be maintained and last over time. Writing code that maintains a consistent use of value/pointer semantics for a given type of data is an important way to achieve this.

Data Oriented Design

“If you don’t understand the data, you don’t understand the problem. This is because all problems are unique and specific to the data you are working with. When the data is changing, your problems are changing. When your problems are changing, the algorithms (data transformations) needs to change with it.” - Bill Kennedy

Think about this. Every problem you work on is a data transformation problem. Every function you write and every program you run takes some input data and produces some output data. Your mental model of software is, from this perspective, an understanding of these data transformations (i.e., how they are organized and applied in the code base). A “less is more” attitude is critical to solving problems with fewer layers, statements, generalizations, less complexity and less effort. This makes everything easier on you and your teams, but it also makes it easier for the hardware to execute these data transformations.

Type (Is Life)

“Integrity means that every allocation, every read of memory and every write of memory is accurate, consistent and efficient. The type system is critical to making sure we have this micro level of integrity.” - William Kennedy

If data drives everything you do, then the types that represent the data are critical. In my world “Type Is Life”, because type provide the compiler with the ability to ensure the integrity of the data. Type also drives and dictates the semantic rules code must respect for the data that it operates on. This is where the proper use of value/pointer semantics starts: with types.

Data (With Capability)

“Methods are valid when it is practical or reasonable for a piece of data to have a capability.” - William Kennedy

The idea of value/pointer semantics doesn’t hit Go developers in the face until they have to make a decision about the receiver type for a method. It’s a question I see come up quite a bit: should I use a value receiver or pointer receiver? Once I hear this question, I know that the developer doesn’t have a good grasp of these semantics.

The purpose of methods is to give a piece of data capability. Think about that. A piece of data can have the capability to do something. I always want the focus to be on the data because it is the data that drives the functionality of your programs. Data drives the algorithms you write, the encapsulations you put in place and the performance you can achieve.

Polymorphism

“Polymorphism means that you write a certain program and it behaves differently depending on the data that it operates on.” - Tom Kurtz (inventor of BASIC)

I love what Tom said in that quote above. A function can behave differently depending on the data it operates on. That the behavior of data is what decouples functions from the concrete data types they can accept and work with. This is one core reason for a piece of data to have a capability. It is this idea that is the cornerstone of architecting and designing systems that can adapt to change.

Prototype First Approach

“Unless the developer has a really good idea of what the software is going to be used for, there’s a very high probability that the software will turn out badly. If the developers don’t know and understand the application well, then it’s crucial to get as much user input and experience as possible.” - Brian Kernighan

I want you to always focus first on understanding the concrete data and algorithms you need to get data transformations working to solve problems. Take this prototype first approach and write concrete implementations that could also be deployed in production (if it is reasonable and practical to do so). Once a concrete implementation is working and once you’ve learned what works and doesn’t work, focus on refactoring to decouple the implementation from the concrete data by giving the data capability.

Semantic Guidelines

You must decide which semantic, value or pointer, is going to be used for a particular data type at the time the type is declared. API’s that accepts or return data of that type must respect the semantic chosen for the type. API’s are not allowed to dictate or change semantics. They must know what semantic is being used for the data and conform to that. This is, at least partially, how consistency across a large code base can be achieved.

Here are the basic guidelines:

At the time you declare a type you must decide what semantic is being used.
Functions and methods must respect the semantic choice for the given type.
Avoid having method receivers that use different semantics than those corresponding to a given type.
Avoid having functions that accept/return data using different semantics than those corresponding to the given type.
Avoid changing the semantic for a given type.

There are a few exceptions to these guidelines with the biggest being unmarshaling. Unmarshaling always requires the use of pointer semantics. Marshaling and unmarshaling seem to always be exceptions to the rule.

How do you chose one semantic over the other for a given type? These guidelines will help you answer the question. Below we will apply the guidelines in certain scenarios:

Built-In Types

Go’s built-in types represent numeric, textual and boolean data. These types should be handled using value semantics. Don’t use pointers to share values of these types unless you have a very good reason.

As an example, look at these function declarations from the strings package.

func Replace(s, old, new string, n int) string
func LastIndex(s, sep string) int
func ContainsRune(s string, r rune) bool

All of these functions use value semantics in the design of the API.

Reference Types

Reference types represent the slice, map, interface, function and channel types in the language. These types should be using value semantics because they have been designed to stay on the stack and minimize heap pressure. They allow each function to have their own copy of the value, as opposed to each function call causing a potential allocation. This is possible because these values contain a pointer that share the underlying data structure between calls.

Don’t use pointers to share values of these types unless you have a very good reason. Sharing a slice or map value down the call stack into an Unmarshal function could be one exception. As an example, look at these two types declared in the net package.

type IP []byte
type IPMask []byte

The IP and IPMask types are both based on a slice of bytes. This means they are both reference types and they should follow the value semantic rules. Here is a method named Mask that was declared for the IP type that accepts a value of IPMask.

func (ip IP) Mask(mask IPMask) IP {
    if len(mask) == IPv6len && len(ip) == IPv4len && allFF(mask[:12]) {
        mask = mask[12:]
    }
    if len(mask) == IPv4len && len(ip) == IPv6len && bytesEqual(ip[:12], v4InV6Prefix) {
        ip = ip[12:]
    }
    n := len(ip)
    if n != len(mask) {
        return nil
    }
    out := make(IP, n)
    for i := 0; i < n; i++ {
        out[i] = ip[i] & mask[i]
    }
    return out
}

Notice that this method is a mutation operation and is using a value semantic API style. It uses a value of IP as the receiver and depending on the IPMask value that is passed in, creates a new IP value and returns a copy of it back to the caller. The method is respecting the fact that you use value semantics for references types.

This is the same for the built-in function append.

var data []string
data = append(data, "string")

The append function is using value semantics for this mutation operation. You pass a slice value into append and after the mutation it returns a new slice value.

The exception is always unmarshaling, which requires pointer semantics.

func (ip *IP) UnmarshalText(text []byte) error {
    if len(text) == 0 {
        *ip = nil
        return nil
    }
    s := string(text)
    x := ParseIP(s)
    if x == nil {
        return &ParseError{Type: "IP address", Text: s}
    }
    *ip = x
    return nil
}

The UnmarshalText method is implementing the encoding.TextUnmarshaler interface. If pointer semantics were not used, it wouldn’t work. But this is ok because it is usually safe to share a value. Outside of unmarshaling, you should raise a flag if pointer semantics are being used for a reference type.

User Defined Types

This is where you will need to make the most decisions. You must decide at the time you are declaring a type what semantic will be used.

What if I asked you to write the API for the time package and I gave you this type.

type Time struct {
    sec  int64
    nsec int32
    loc  *Location
}

What semantic would you use?

Look at the implementation of this type in the Time package along with the factory function Now.

func Now() Time {
    sec, nsec := now()
    return Time{sec + unixToInternal, nsec, Local}
}

A factory function is one of the most important functions for a type because it tells you what semantic is being chosen. The Now function is making it clear that value semantics are in play. This function creates a value of type Time and returns a copy of this value back to the caller. Sharing Time values is not necessary and they don’t need to end up on the heap.

Also, look at the Add method, which is a mutation operation.

func (t Time) Add(d Duration) Time {
    t.sec += int64(d / 1e9)
    nsec := t.nsec + int32(d%1e9)
    if nsec >= 1e9 {
        t.sec++
        nsec -= 1e9
    } else if nsec < 0 {
        t.sec--
        nsec += 1e9
    }
    t.nsec = nsec
    return t
}

Again you can see that the Add method is respecting the semantic chosen for the type. The Add method is using a value receiver to operate on its own copy of the Time value, where that copy of the Time value is being used to make the call. It then mutates its own copy and returns a new copy of the Time value back to the call.

Here is a function that accepts Time values:

func div(t Time, d Duration) (qmod2 int, r Duration) {

Once again, value semantics are being used to accept a value of type Time. The only use of pointer semantics for the Time API are these Unmarshal related functions:

func (t *Time) UnmarshalBinary(data []byte) error {
func (t *Time) GobDecode(data []byte) error {
func (t *Time) UnmarshalJSON(data []byte) error {
func (t *Time) UnmarshalText(data []byte) error {

Most of the time your ability to use value semantics is limiting. It isn’t correct or reasonable to make copies of the data as it passes from function to function. Changes to the data need to be isolated to a single value and shared. This is when pointer semantics need to be used. If you are not 100% sure it is correct or reasonable to make copies, then use pointer semantics.

Look at the factory function for the File type in the os package.

func Open(name string) (file *File, err error) {
    return OpenFile(name, O_RDONLY, 0)
}

The Open function is returning a pointer of type File. This means you should be using pointer semantics and always share File values. Changing the semantic from pointer to value could be devastating to your program. When a function shares a value with you, you should assume that you are not allowed to make a copy of the value pointed to by the pointer. If you do, results will be undefined.

Looking at more of the API you will see the consistent use of pointer semantics.

func (f *File) Chdir() error {
    if f == nil {
        return ErrInvalid
    }
    if e := syscall.Fchdir(f.fd); e != nil {
        return &PathError{"chdir", f.name, e}
    }
    return nil
}

The Chdir method uses pointer semantics even though the File value is never mutated. The method must respect the semantic convention for the type.

func epipecheck(file *File, e error) {
    if e == syscall.EPIPE {
        if atomic.AddInt32(&file.nepipe, 1) >= 10 {
            sigpipe()
        }
    } else {
        atomic.StoreInt32(&file.nepipe, 0)
    }
}

Here is a function named epipecheck and it’s using pointer semantics to accept the File value. Again, notice the consistent use of pointer semantics for values of type File.

Conclusion

The consistent use of value/pointer semantics is something I look for in code reviews. It helps you keep code consistent and predictable over time. It also allows everyone to maintain a clear and consistent mental model of the code. As the code base and the team gets larger, consistent use of value/pointer semantics becomes even more important.