Defeat pointers in C and Go

Table of Contents

Introduction #

It is a common opinion that “pointers are difficult”, however, as we will see in this article, they aren’t rocket science. In my view they are:

• easy: you just need a very simple model of how computers work to be able to understand pointers

• powerful: pointers expand what is possible to do with functions and they allow to build any desired data structure

• fun: If you like to understand how things are implemented and what is the magic behind the scenes, you will love to play with pointers

How computers work #

I assume you are already familiar with a higher level programming language, so we know the two main components of a computer are:

1. the processor: a circuit that allows the execution of the operations specified in a program

2. the memory: another circuit with the purpose of storing data

Let’s get a little bit deeper and look in more detail at what they are capable of and how they interact with each other.

The processor has the capability of performing mathematical operations between values previously loaded into its registers (memory cells implemented directly in the hardware). From a programmer’s perspective we can think about registers as a predefined set of allocated variables named R1, R2, R3, etc. The number of registers is in general not sufficient to hold all the variables in our programs.

On the other hand the memory does not have computational power by itself, but can hold much more data than the processor. We can think about memory as a set of cells with a size of 1 byte. To identify each memory location we use a number called address of the cell. Usually addresses are big numbers so it is more convenient to express them in hexadecimal format. If you ever used an array you don’t have to learn anything new: the memory is a long array of bytes and the address is simply the number of the element we want to access.

Processor and memory are connected to each other so the processor can retrieve values from memory and store values into memory. A simple program to calculate the sum of 2 numbers looks like:

# Processor Pseudo-code
R1 = read_byte_from_memory_at_address(0x1200)
R2 = read_byte_from_memory_at_address(0x1300)
R3 = R1 + R2
store_byte_into_memory_at_address(R3, 0x2000)

The usage of the hexadecimal number for the address (and the prefix 0x) makes it clear for the reader when a number is an address.

Pointers and Addresses #

We can start defining a pointer as a variable holding an address of another variable. This is true, but not a complete definition. Moreover you could think “why do we need another data type just for that purpose? As we have seen previously, an address is an integer number, both in C and in Go we already have a variable type for it.” The problem is that the address is sufficient only if we store our value in one byte, but to store for example the number 512 we need at least 2 bytes. So when we say “the value 512 is stored at address 0x1200”, we are actually saying “the value 512 is stored in the cell 0x1200 and 0x1201 of the memory”. Therefore a pointer is a variable that can hold an address of a value and it knows how many bytes are used to store that value.

Unless you are working on specific applications, usually it is not the job of a programmer to decide where each variable is stored, but the compiler and the operating system take care of it. This process is called memory allocation and it takes place every time we define a new variable or we use a data structure that increases (or decreases) its memory occupation.

To start using pointers and inspect the behavior of some programs, we need to learn how to do the following tasks:

• declare a pointer
• get an address of a variable
• access the memory location addressed by a pointer

The name pointer comes from the sentence "p points to a" which is used to say quickly “the content of p is the address of a”.

Syntax of pointers in C and Go #

To declare a pointer we add an asterisk in the declaration of the variable. For example in the following listing we are declaring two integers and two pointers:

// C language
int a;     // Declaration of an integer variable
float b;   // Declaration of a float variable

int *p1;    // Declaration of a pointer to an integer variable
float *p2;  // Declaration of a pointer to a float variable

// Go language
var a int     // Declaration of an integer variable
var b float   // Declaration of a float variable

var p1 *int    // Declaration of a pointer to an integer variable
var p2 *float  // Declaration of a pointer to a float variable

Note that in C also the following declarations are perfectly valid:

int* p1;
int * p1;

but the idiomatic way of declaring pointers in C is by attaching the asterisk to the variable name.

Both in C and in Go we can use the operator & to get the address of a variable, and the operator * to access the pointed memory location.

The first source of confusion (in my opinion) is because the same operator * is used in different contexts:

• to declare a new pointer
• to access the memory location

Let’s see the following fragment of codes:

// C language
int a = 4;
int *p;              // asterisk here is used to declare a pointer
p = &a;              // assign the address of a to p
printf("%d\n", *p);  // asterisk here is used to get the pointed value

// Go language
a := 4
var p *int           // asterisk here is used to declare a pointer
p = &a               // assign the address of a to p
fmt.Println(*p)      // asterisk here is used to get the pointed value

Whenever you see an asterisk, ask yourself “is this a declaration?”, if so, the asterisk means “we are declaring a pointer”, otherwise it can be an access to a pointed variable. In the latter case don’t forget it could be also the multiplication of two variables.

The empty value for pointers #

Okay, we understand now what happens when a pointer holds the address of another variable, but what happens exactly when I have just declared my pointers? I haven’t assigned any value to them yet.

In C there is no automatic initialization of variables, so to indicate that a pointer does not hold a valid address we can assign the value NULL (which is zero). The best practice here is to assign NULL to the variable during declaration or to provide an initial value:

// C language
int *p = NULL;
int *q = &a;

In Go the empty pointer holds the value nil and it is automatically assigned when a new variable is declared.

How to analyze programs with pointers #

Before discussing real world applications of pointers, it’s better to become a little bit more familiar with them. Let’s run this program and see what’s printed on the screen:

// C language
int a = 4;
int *p;
p = &a;
printf("address_of_a=%p   p=%p\n", &a, p);
printf("a=%d   value_pointed_from_p=%d\n", a, *p);
printf("address_of_p=%p\n", &p); /* p is a variable so it has an address itself */

// Go language
a := 4
var p *int
p = &a
fmt.Printf("address_of_a=%p   p=%p\n", &a, p)
fmt.Printf("a=%d   value_pointed_from_p=%d\n", a, *p)
fmt.Printf("address_of_p=%p\n", &p) /* p is a variable so it has an address itself */

The addresses of the variables can change from execution to execution, but the first print statement should have two equal numbers and the second one must always contain a couple of 4

Output 1

address_of_a=0x1200     p=0x1200

a=4     value_pointed_from_p=4

address_of_p=0x1300

The third print is just to spot a second point of confusion:

• a pointer is a variable, so it is allocated at a certain address in memory
• a pointer contains the address of another variable

Confusion might arise if we are not clear about which address we are considering. However it is almost always the address held by the pointer.

A useful tool: the memory table #

When we analyze a program with pointers it is useful to keep track of addresses and values while mentally executing the code line by line. Consider the following:

// C language
int a=3;
int b=1;
int *p=&a;
int *q=&a;
b = b + 1;
a = (*p + b) * a - *q;
printf("a=%d b=%d\n", a, b);
q = &b;
*p = a + b + *p + *q;
printf("a=%d b=%d\n", a, b);

// Go language
a:=3
b:=1
p:=&a
q:=&a
b = b + 1
a = (*p + b) * a - *q
fmt.Printf("a=%d b=%d\n", a, b)
q = &b
*p = a + b + *p + *q
fmt.Printf("a=%d b=%d\n", a, b)

Let’s build a simple table with a column for the addresses, one for the names of variables and one for their contents:

Now we will read our program line by line and we will fill the memory table:

• The program starts with the declaration of variable a so we are going to put a in the first row in the second column.
• Let’s say the address of a is 0x1200. Put it in the first column of the first row. It doesn’t matter if it is invented, but once we have decided the value of an address we need to be consistent with it: every time the program will use the address of a we have to use this value.
• Put 3 in the last column because this is the value assigned to a by the first line of the code.
• At the second line there is the declaration of b, so we can do the same and fill out another row of the table. Let’s use 0x1300 for the address of b (and don’t forget 1 in the third column for the value).
• In the third line the program declares the pointer p and the address of a is assigned to it. We have all we require to add a new row in the table (let’s use 0x1400 for the address where p is allocated). Note that the content of p is 0x1200 since it is the address of a.

At this point my table looks like this:

Let’s continue to use the memory table as the current snapshot of our memory while executing the program line by line. If the value of a variable changes, simply delete the old value and update the number in the third column.

After the execution of b = b + 1 my table is:

As you can see, b had a value of 1 at first, but now I have changed the original value to 2.

The following line in the program is more challenging, but you just have to analyze the expression breaking it into small pieces:

• keep the assignment as the last thing to do
• the first piece of expression we need to compute is *p + b. In this expression there are actually two operations: a sum and an access to the pointed value. If I had to read this would be: “get the value pointed by p and add it to b”. So I check the table and see the value of p is 0x1200, so I jump to the line of the table with that address (first row) and move my eye in the same row up to the third column to check what is the value: 3. Now let’s add 3 to the value of b (I read the value of b in the second row of the table) and got: 5.
• now we have to compute the rest of the expression which is 5 * a - *q. With the help of the table, the value of a is 3 and the value pointed by q is 5, so 5*3-5 = 10
• the right-hand side of the equal is 10, therefore we need to assign 10 to a. Let’s update the value of a in the table:

The next line prints the content of a and b so, since we have 10 and 5 in our table, the first line of the output of our code will be:

Output

a=3 b=13

As an exercise, please continue reasoning like this to compute what will be the second line of the output. Then you can verify it by compiling and running the program. It is hard to get it right the first time, so don’t be discouraged if you weren’t able to solve it correctly.

If you need practice, you can go on writing a new invented code, creating the memory table and then verifying the output.

Usage of pointers in functions #

Both C and Go use the passing by value model when a variable is passed to a function. Let’s see an example of what it means in practice. Assume we want to refactor the codes of this program extracting a couple of functions:

// C language
int main(int argc, char**argv){
    int a = 2;
    int b = 4;
    printf("a=%d b=%d\n",a,b);

    a = a * 3;  // triple the value in a
    printf("a=%d b=%d\n",a,b);

    /* swapping the content of two variables */
    int t = a;
    a = b;
    b = t;
    printf("a=%d b=%d\n",a,b);
}

// Go language
func main() {
    a := 2
    b := 4
    fmt.Printf("a=%d b=%d\n",a,b)

    a = a * 3  // triple the value in a
    fmt.Printf("a=%d b=%d\n",a,b)

    /* swapping the content of two variables */
    t: = a
    a = b
    b = t
    fmt.Printf("a=%d b=%d\n",a,b)
}

Output 1

a=2 b=4

a=6 b=4

a=4 b=6

A new programmer might be tempted to break it like this:

// C language
void triple(int a){
    a = a * 3;
}

void swap(int a, int b){
    /* swapping the content of two variables */
    int t = a;
    a = b;
    b = t;
}

int main(int argc, char**argv){
    int a = 2;
    int b = 4;
    printf("a=%d b=%d\n",a,b);

    triple(a);
    printf("a=%d b=%d\n",a,b);

    swap(a,b);
    printf("a=%d b=%d\n",a,b);
}

// Go language
func triple(a int) {
    a = a * 3
}

func swap(a int, b int) {
    /* swapping the content of two variables */
    t: = a;
    a = b;
    b = t;
}

func main() {
    a := 2
    b := 4
    fmt.Printf("a=%d b=%d\n",a,b);

    triple(a);
    fmt.Printf("a=%d b=%d\n",a,b);

    swap(a,b)
    fmt.Printf("a=%d b=%d\n",a,b);
}

But in this case the output will be:

Output 2

a=2 b=4

a=2 b=4

a=2 b=4

The problem here is that the language passes the parameters by value, so when we are tripling the content of a, we are actually tripling the content of a copy of a (this became apparent if you call the parameter of the function with different names). The original variable is not touched. Let’s analyze the code with the tool we learned in the previous section the memory table. It might be useful also to add more print statements in the code at the end of every function (and to edit those in the main):

• use printf("&a=%p a=%p &b=%p b=%d\n",&a,a,&b,b); in the C code
• use fmt.Printf("&a=%p a=%p &b=%p b=%d\n",&a,a,&b,b) in the Go code

The memory table we get should be something similar to:

Since in the code corresponding to Output 2 the print statements are only in the main function, it prints only the values of the first two rows of our table.

How do we fix it?

The simplest solution in Go is to return the newly computed value and add an assignment in the main:

// Go language
func triple(a int) int {
    a = a * 3
    return a
}

func swap(a int, b int) (int, int) {
    /* swapping the content of two variables */
    t: = a;
    a = b;
    b = t;
    return a, b
}

func main() {
    a := 2
    b := 4
    fmt.Printf("a=%d b=%d\n",a,b);

    a = triple(a);
    fmt.Printf("a=%d b=%d\n",a,b);

    a, b = swap(a,b)
    fmt.Printf("a=%d b=%d\n",a,b);
}

Output 3

a=2 b=4

a=6 b=4

a=4 b=6

In C we can use the same solution for triple(), but since there is no possibility of returning two independent variables from a function the only workaround that does not involve pointers is to pack the 2 variables in a struct, return the struct and unpack it in the main. For brevity we will skip this solution.

Let’s see how using pointers we will be able to directly change the values of a and b declared in the main:

// C language
void triple(int *a){
    *a = (*a) * 3;
}

void swap(int *a, int *b){
    /* swapping the content of two variables */
    int t = *a;
    *a = *b;
    *b = t;
}

int main(int argc, char**argv){
    int a = 2;
    int b = 4;
    printf("a=%d b=%d\n",a,b);

    triple(&a);
    printf("a=%d b=%d\n",a,b);

    swap(&a,&b);
    printf("a=%d b=%d\n",a,b);
}

// Go language
func triple(a *int) {
    *a = (*a) * 3
}

func swap(a *int, b *int) {
    /* swapping the content of two variables */
    t: = *a;
    *a = *b;
    *b = t;
}

func main() {
    a := 2
    b := 4
    fmt.Printf("a=%d b=%d\n",a,b);

    triple(&a);
    fmt.Printf("a=%d b=%d\n",a,b);

    swap(&a,&b)
    fmt.Printf("a=%d b=%d\n",a,b);
}

Output 4

a=2 b=4

a=6 b=4

a=4 b=6

Memory allocation and dynamic data structures #

In the introduction we said pointers are very useful to implement all the data structures we need. In C the only data structure handled by the language is an array of fixed size. All the others need to be implemented.

In Go we have arrays of fixed size (rarely used), slices and maps. The latter two are dynamic data structures because the memory allocated can grow or reduce depending on the data we are inserting or deleting.

That being said, there is a certain number of nonobvious behaviors in slices. Of course a programmer can just learn a bunch of rules like “never append a value to a slice received as a parameter in a function”, but why is that? The reason becomes apparent when we try to guess how it is implemented.

In order to proceed with a possible implementation of slices, we need to discuss first how the memory is managed in each of the two languages.

Memory management in C #

In C the compiler manages automatically the allocation (and the release) of all the variables declared in a function using a part of the memory called stack. To perform this job the size of each variable need to be known at compile time. This is the reason why the language does not support any dynamic data structure. However in the situations where we need to allocate a different number of values depending on the input of the program, we can use the functions malloc() and free(). They interact with the Operating Systems to allocate and release memory at run time.

To create for example a new integer variable at run-time we can use the following instructions:

// C language
int *p;
p = malloc(sizeof(int));

where sizeof(int) is the number of bytes required to store an integer.

The function malloc() returns the address of the new allocated variable so, in this case, we must use a pointer to save its location. The new variable lives in a region of the memory called heap and it remains allocated until we explicitly call free().

As a consequence of this way of managing the memory, it is an error in C to return the address of a local variable:

// C language
int *wrong_function(){
    int a;
    return &a; /* ERROR: the address returned by this function will
                *        allow another part of the program to access
                *        the old location of "a". This location will
                *        be probably reused by another variable and
                *        will cause failure in the program in a way
                *        very difficult to debug
                */
}

int *correct_function(){
    int *p = malloc(sizeof(int));
    return p;
}

int main(){
    int *wrong_pointer = wrong_function();
    /* ERROR: the address in wrong_pointer is not valid anymore
     *        because we are out of the scope of wrong_function
     */
    int *q = correct_function();
    *q = 4; /* This is perfectly safe because the memory allocated by
             * malloc remains valid until we call free()
             */
    free(q);
    // if we try to access q here it would point to a not valid memory
    // location. For this reason better to put a NULL in the pointer.
    q = NULL;

    return 0;
}

Memory management in Go #

In Go the memory is fully managed by the language. There is a part of the go run-time which is called Garbage Collector. This is in charge of releasing the allocated memory when it is not referenced by any pointer or variable. For this reason there is no need to call a function similar to free() and it is perfectly safe to return an address of a local variable [1]:

// Go language
func correct_function() *int {
     var a int
     return &a // This is C would be an error, but in Go works well
}

func main() {
     var q *int
     q = correct_function()
     *q = 4
}

Dynamic Arrays in C, slices in Go #

Let’s see how we can implement a dynamic array of integers in C:

// C language
struct dynamic_array {
    int len; /* length of the array: keep track of how many elements
                we have put in the array */
    int cap; /* capacity of the array: allocated elements available to
                store data */
    int *v;  /* pointer to the allocated array */
}

struct dynamic_array free_dyn_array(struct dynamic_array d){
    ... /* error checking skipped for brevity */
    free(d.v);
}

struct dynamic_array copy(struct dynamic_array d_dest, struct dynamic_array d_src){
    ... /* error checking skipped for brevity */
    for(i=0; i < d_src.len; i++)
        d_dest.v[i]=d_src.v[i];
    return d_dest;
}

struct dynamic_array make(struct dynamic_array d, int len, int cap){
    ... /* error checking skipped for brevity */
    d.len = len;
    d.cap = cap;
    d.v = malloc(sizeof(int)*d.cap);
    return d;
}

struct dynamic_array append(struct dynamic_array d, int val) {
    ... /* error checking skipped for brevity */
    if(d.len < d.cap) {
        d.v[d.len] = val;
        d.len = d.len + 1;
        return d;
    } else {
        struct dynamic_array new_d;
        new_d = make(new_d, d.len, d.cap+1);
        new_d = copy(new_d, d);
        free_dyn_array(d);
        return new_d
    }
}

Note that there are several possible implementations for dynamic arrays in C (check for example the source code of the Apache Portable Library[4] or of GLib[5]). Our implementation is done in a way that recalls how slices are probably implemented in Go:

• we used the make() function to create a dynamic array
• we used the append() function to insert a new element in the array
• we used copy() to replicate the content of a dynamic array in another one

The code above is useful to understand the nonobvious behaviors of slides. For example the following should now be clarified:

• when a slice is passed to a function and the parameter of the function changes the value in the slice, sometimes it changes the value in the original slice, sometimes it doesn’t.
• when a slice is copied with = and the values at the second slice are changed, sometimes also the first slice reflects the changes, sometimes it doesn’t.

Please read the chapter in the book [3] or wait until I update this article.

References #

[1] https://stackoverflow.com/questions/52996452/returning-pointer-from-a-local-variable-in-function

[2] https://go.dev/blog/slices-intro

[3] Learning Go - an idiomatic approach, Jon Bodner, 2021, O’Reilly

[4] https://apr.apache.org/docs/apr/trunk/group__apr__tables.html

[5] https://docs.gtk.org/glib/data-structures.html#arrays<