Posts tagged 'golang'

Build and run Golang projects in VS Code

published on January 24, 2018.

I’ve been using VS Code for my Golang development needs for a few months now. Minor kinks here and there, nothing serious, and the development experience gets better with every update. I have also tried out IntelliJ Idea as the editor, and one feature that I’m missing in Code from Idea is the build-run-reload process. I thought to myself, that’s such a basic feature, it should be possible to have that.

And it is! VS Code Tasks to the rescue!

These tasks allow us to run different kind of tools and, well, tasks inside VS Code.

Go to Tasks -> Configure Default Build Task and then select the “Create tasks.json file from template” in the little pop-up window, and after that select the “Others” option. This tasks.json file will live inside the .vscode directory.

For my overcomplicated d20 roller, which is my first website built with Golang, I have the following content for the tasks:

.vscode/tasks.json

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "Build and run",
            "type": "shell",
            "command": "go build && ./d20",
            "group": {
                "kind": "build",
                "isDefault": true
            }
        }
    ]
}

What this one task does is that it runs go build to build the project and then runs the generated executable, which for this project is d20.

I guess providing a standardized name to go build with the -o flag this could be made more portable so that the command part reads something like go build -o proj && ./proj, but I’m ok with this for now.

And now just type Ctrl+Shift+b and Code will execute this “Build and run” task for us! Hooray! The terminal window in Code will pop-up saying something like:

> Executing task: go build && ./d20 <

By going to http://localhost:8080 I can see that my little website is up and running. Cool.

But if we want to restart this taks, running Ctrl+Shift+b again won’t work and Code will complain that the “Task is already active blablabla…”.

Looking at the Tasks menu, we can see that there’s a “Restart running task…” menu entry. Clicking that, it pops up a window with a list of running tasks. In this case there’s only one, our “Build and run” task. Clicking through the menu every time would be boring, so let’s add a keyboard shortcut for it.

Go to File -> Preferences -> Keyboard shortcuts (or just hit Ctrl+k Ctrl+s), search for “Restart running task” keybinding, and set it to whatever you like. I’ve set it to Ctrl+Alt+r.

Finally, the flow is Ctrl+Shift+b to start the taks for the very first time, code-code-code, Ctrl+Alt+r to re-build. Sweet. Now the only annoying bit is that I have to pick out that one running task from the list of running tasks when restarting, but I can live with that. For now.

Happy hackin’!

Buffered vs. unbuffered channels in Golang

published on December 25, 2017.

As any newcomer to Golang and it’s ecosystem, I was eager to find out what is this hubbub about these things called goroutines and channels. I read the documentation and blog posts, watched videos, tried out some of the “hello world” examples and even wasted a couple of days trying to solve the puzzles for day 18 from Advent of Code 2017 using goroutines and failed spectacularly.

All this was just… doing stuff without actually understanding of when, how, and why use goroutines and channels. And without that basic understanding, most of my attempts at doing anything that resembles a useful example ended up with deadlocks. Lots and lots and lots of deadlocks.

Ugh. I decided to stop forcing myself to understand all of this concurrency thing and figure it all out once I actually need it.

Few days later, I start reading the description for the first puzzle day 20. The gist of the problem is that we have many particles “flying” around on three axes: X, Y, and Z. Each particle has a starting position, a starting velocity, and a constant acceleration. When a particle moves, the acceleration increases the velocity of the particle, and that new velocity determines the next position of the particle. Our task is to find the particle that will be the closest to the center point after every particle has moved the same number of times.

Then there’s this one sentence in the description:

Each tick, all particles are updated simultaneously.

Could it… could it be? A puzzle where goroutines can be applied to do what they were meant to be doing? Only one way to find out, and that’s to try and use goroutines to solve the puzzle.

Prior knowledge

Up until this point I knew the following things about goroutines and channels:

  • goroutines are started with the go keyword,
  • we must wait on the gouroutines to be done, otherwise they fall out of scope and things sort of leak,
  • the waiting can be done with either sync.WaitGroup or with a “quit” channel,
  • I prefer the sync.WaitGroup approach as it is easier for me to follow it,
  • goroutines communicate through channels, by sending and receiving a specific data type on them,
  • there are buffered and unbuffered channels, but other than it’s pretty much ¯\_(ツ)_/¯
  • I think it’s not possible to send and receive to the same channel in the same goroutine?

Setting up the code

From the puzzle’s description we know we have a bunch of particles, every particle has an ID from zero to N, every particle has a position, velocity, and acceleration, and we want to know the particle’s destination from the center. I went ahead and made a struct like this:

type Particle struct {
	id int
	p  coord
	v  coord
	a  coord
	d  int
}

The coord is again a struct that holds the XYZ coordinates and has two methods on it, nothing fancy. When the particle moves, we add the acceleration to the velocity and the velocity to the position. Again, nothing interesting there.

That’s pretty much all the setup I had.

Time to move

With all the prior knowledge I had, my idea was to move every particle in a separate goroutine for 10000 times, send every particle over a channel to somewhere where I can compare them and find the one that’s the closest to the center, and use sync.WaitGroup to orchestrate the waiting on the goroutines to move the particles. The 10k is just a random number that seemed high enough so all particles are given enough time to move.

The first iteration looked like something the following:

func closest(particles []Particle) Particle {
	var cp Particle
	var wg sync.WaitGroup
	wg.Add(len(particles))
	pch := make(chan Particle)

	for _, particle := range particles {
		go move(particle, 10000, pch, &wg)
	}

	wg.Wait()
	close(pch)
    return cp
}

cp will hold the closest particle, there’s the wg to do all the waiting, the pch channel to send particles over it.

For every particle in the slice of particles I tell it to “split out” in a goroutine and move that Particle for 10000 times. The move function moves the particle for the given number of iterations, sends it over the pch channel once we’re done moving it, and tells the wait group that it’s done:

func move(particle Particle, iterations int, pch chan Particle, wg *sync.WaitGroup) {
	for i := 0; i < iterations; i++ {
		particle.move()
	}
	pch <- particle
	wg.Done()
}

So far this seems like something that could work. Running this code as is, results in “fatal error: all goroutines are asleep - deadlock!“. OK, sort of make sense that it fails, we only send to the particle channel, we never receive from it.

Send and you shall receive

So… let’s receive from it I guess:

func closest(particles []Particle) Particle {
    // snip...
	for _, particle := range particles {
		go move(particle, 10000, pch, &wg)
	}

    p := <-pch
    log.Println(p)

	wg.Wait()
	close(pch)
    return cp
}

Surprise! “fatal error: all goroutines are asleep - deadlock!“. Errr…

Oh, right, can’t send and receive to a channel in the same goroutine! Even though, receiving is not in the same goroutine as sending, lets see what happens if we do receive in one:

func closest(particles []Particle) Particle {
    // snip...
	for _, particle := range particles {
		go move(particle, 10000, pch, &wg)
	}

    go func() {
        p := <-pch
        log.Println(p)
    }()

	wg.Wait()
	close(pch)
    return cp
}

Guess what? “fatal error: all goroutines are asleep - deadlock!” This should totally be possible, I’m doing something wrong!

Matej mentioned something on Twitter the other day that buffered channels can send and receive inside the same goroutine. Let’s try a buffered channel, see if that works.

Maybe buffers is what we need after all

When creating the pch channel, we pass in a second argument to make, the size of the buffer for the channel. No idea what’s a good size for it, so let’s make it the size of the particles slice:

func closest(particles []Particle) Particle {
    // snip...
	pch := make(chan Particle, len(particles))

	for _, particle := range particles {
		go move(particle, 10000, pch, &wg)
	}

    p := <-pch
    log.Println(p)

	wg.Wait()
	close(pch)
    return cp
}

Run it and… A-ha! One particle gets logged! There’s no comparing of particles in there yet, so it must be the first particle that was sent on that channel. OK, let’s range over it, that’ll do it:

func closest(particles []Particle) Particle {
    // snip...
	for _, particle := range particles {
		go move(particle, 10000, pch, &wg)
	}

    for p := range pch {
        log.Println(p)
    }

	wg.Wait()
	close(pch)
    return cp
}

“fatal error: all goroutines are asleep - deadlock!” motherf… gah! Fine, I’ll loop over all the particles and receive from the channel, see what happens then:

func closest(particles []Particle) Particle {
    // snip...
	for _, particle := range particles {
		go move(particle, 10000, pch, &wg)
	}

    for i := 0; i < len(particles); i++ {
		p := <-pch
		log.Println(p)
	}

	wg.Wait()
	close(pch)
    return cp
}

Bingo! All particles logged, no deadlocks. Throw in a closure to find the closest particle and we’re done!

// snip...
    var findcp = func(p Particle) {
        if cp.d == 0 || particle.d < cp.d {
			cp = particle
		}
    }
    for i := 0; i < len(particles); i++ {
		p := <-pch
		findcp(p)
	}
// snip...

For my input I get that the answer is Particle number 243, submit it to Advent of Code and it’s the correct answer! I did it! I used goroutines and channels to solve one puzzle!

But… how?

I made it work, but I still didn’t understand how and why does this work. Time to play around it with some more.

I’ve seen code examples using range to range over a channel and use whatever is received from the channel to do something with it. Countless blog posts and tutorials, I’ve never seen a “regular” for loop and in it receiving from the channel. There must be a nicer way to achieve the same. Re-reading a couple of the articles, I spot the error I made in the range approach; I was closing the channel too late:

func closest(particles []Particle) Particle {
    // snip...
	for _, particle := range particles {
		go move(particle, 10000, pch, &wg)
	}

	wg.Wait()
	close(pch)

    for p := range pch {
        findcp(cp)
    }

    return cp
}

Particle number 243! After some thinking about it, it makes sense, or at least this is how I made it make sense:

Lesson number 1 — golang’s range doesn’t “like” open-ended things, it needs to know where does the thing we range over begin and where does it end. By closing the channel we tell range that it’s OK to range over the pch channel, as nothing will send to it any more.

To put it in another way, if we leave the channel open when we try to range over it, range can’t possibly know when will the next value come in to the channel. It might happen immediately, it might happen in 2 minutes.

Next step is to try and make it work using unbuffered channels. If I just make this buffered channel an unbuffered one, but otherwise leave the working code as-is, it blows up with a deadlock. Something something same goroutine.

Back to the example where I tried to receive in a separate goroutine:

    // snip..
    go func() {
        p := <-pch
        log.Println(p)
    }()
    // snip...

Ah, this goroutine runs only once. It even prints out a single particle at the beginning, I just didn’t look closely enough the first time, all I saw was the deadlock error message and moved on. I should probably loop over the channel somehow and receive from it in that loop.

I remembered reading something about a weird looking for/select loop, let’s try writing one of those:

    // snip..
    go func() {
        for{
			select {
			case p := <-pch:
				findcp(p)
			}
		}
    }()
    // snip...

Particle number 243! Yey! And again, after giving it some thought, this is how I explained this unbuffered channel version to myself:

Lesson number 2 — an unbuffered channel can’t hold on to values (yah, it’s right there in the name “unbuffered”), so whatever is sent to that channel, it must be received by some other code right away. That receiving code must be in a different goroutine because one goroutine can’t do two things at the same time: it can’t send and receive; it must be one or the other.

So far whatever I threw at these two lessons learned, they’re standing their grounds.

Buffered or unbuffered?

Armed with these two bits of new knowledge, when to use buffered and when to use unbuffered channels?

I guess buffered channels can be used when we want to aggregate data from goroutines and then do with that data some further processing either in the current goroutine or in new ones. Another possible use case would be when we can’t or don’t want to receive on the channel at the exact moment, so we let the senders fill up the buffer on the channel, until we can process it.

And I guess in any other case, use an unbuffered channel.

And that’s pretty much all I learned from this one Advent of Code puzzle. Here’s my final solution using buffered channels and here’s the one using unbuffered channels. I even figured out other ways to make it work while writing this blog post, but those solutions all come from understanding of how channels work.

If you spot any errors in either my working code examples or in my reasoning, please let me know. I want to know better. Thanks!

Happy hackin’!

Five days of Advent of Gode

published on December 06, 2017.

A week or so ago, Luka mentioned this Advent of Code thing. I’ve been doing coding challenges and examples before, but never have I tried the AoC (this is the third year it’s running).

Advent of Code is a series of programming puzzles, where you get 2 puzzles a day for 25 days.

Given that I have started to learn golang again, I figured might as well learn more about it by joining this years challenge.

I keep a repository with my solutions to these puzzles and try to take notes for every day and now I want to look back at the first 5 days.

I started of the first day with just a mess of a code, just pushing for the first solution that gets the correct answer. On the second day I realised I won’t be learning much like this, so I decided to bring some order to the chaos: organised the code a bit nicer (even though it’s a far cry from good), added tests, and generally tried to get to a point where it’s easier to get started with a day’s challenge.

The AoC puzzles are relatively easy so far. Day 3 was the only day so far where I had problems wrapping my head around the problem. I’ve managed to figure out the solution for the first puzzle, but for the second puzzle I “cheated” and used the OEIS. shrug

As for golang… I don’t know enough of it to say if I like it or not. I mean, I do like it, sort of, but for these 5 days the most I did was toying around with strings and integers and slices and maps. That’s hardly enough time and usage to pass judgment on it.

On golang

I like how it fails to build if there’s an unused variable laying around, but then again I tend to save often, so I write:

for k, v := range slc {
}

Hit save and the IDE underlines that entire for line. What, why?! Hover over the line to see what the problem is… Oh, k and v are unused. Well, d-uh, I’m not done yet. But I still have to double check because the bug might be real, for example if I want to range over an integer the IDE will again underline the entire line.

For some reason I’m really bad at naming things in golang, quite often the variable and function names are just bad. I’m trying my best to follow the golang styles, but… I don’t know. I’m not sold on the whole “short rather than long” thing.

I keep mixing assignment = and assignment & declaration :=. But I’ll learn it, eventually.

I like everything about the types, even though I don’t understand everything about them, yet. Maps, for example, are not ordered even though I tried to use them as such once.

On slices

Except slices. Slices are weird. Well, were weird until I understood how golang treats and works with them.

I read four different articles to get to the point where I know what a slice is, only to get a succinct explanation of “slices are mutable views of an array”. For me, that one sentence explains it better than the four articles.

Here’s an example:

package main

import (
	"fmt"
)

func main() {
	x := []int{1, 2, 3, 4, 5}
	a := make([]int, 0)
	b := make([]int, len(x))

	y := x
	copy(a, x)
	copy(b, x)

	F(x)

	fmt.Println(x) // [2 3 4 5 6]
	fmt.Println(y) // [2 3 4 5 6]
	fmt.Println(a) // []
	fmt.Println(b) // [1 2 3 4 5]
}

func F(z []int) {
	for k, _ := range z {
		z[k]++
	}
}

Notice how both x and y are the same, even though we only modify x within the F() function, without even returning anything from that function. What happens is that the array on which the slice is built gets modified, which in turns modifies the slice(s) as well. Another thing worth remembering is to only append to a slice with the append() function.

On testing

I learned how to write tests. I guess there’s more to learn about them, but so far I’m doing OK. I’m writing a lot of repetitive things like this to get data providers:

for _, tt := range footests {
    r := Foo(tt.in)

    if r != tt.out {
        t.Errorf("Got %d for %s, expected %d", r, tt.in, tt.out)
    }
}

Even if there’s no shorter/better way to handle data providers in golang tests, I’ll just create some snippets and be done with all the copy/pasting.

I like that VS Code allows to run and debug a single test function. That’s really helpful. It shows small “run test” and “debug test” links above every test method. Click and off we go!

On packages

I haven’t yet learned how to properly work with my own packages, how to organize code and name packages to import one into the other, but I didn’t really had the need for it.

What I did learn, is that I can’t name a function like an existing package. For example, I wrote my own little function called sort, which made it impossible to import the sort package from the standard library.

All in all, I’m making good progress with both the AoC puzzles and with golang. I believe this will give me a good foundation for further learning and improving. There’s still a long road before me, but I feel I’m on the right track.

Happy hackin’!

Reading from standard input with Go

published on October 30, 2017.

Last year I started learning Go, then I stopped, and now I don’t remember anything what I learned.

Going to try it a bit different this time, by writing down what I do, learn, experiment.

As I don’t have a specific thing I want to build with Go, I’m just going to do simple scripts and examples, to get to know Go’s language specification and built-ins as much as possible.

Hopefully, there won’t be too much mistakes in my code samples. I’ve set up a golearn repository just for this, so feel free to create issues/PRs telling me where I went wrong.

Setting up the Go environment

I installed Go to the location that the official docs recommend — /usr/local, and I created the “Go workspace” in ~/projects/gocode and then I put my code under ~/projects/gocode/src/github.com/robertbasic/golearn.

I do find it kinda weird that everything lives in one “workspace”. But, whatever.

As for the editor, I’m using Visual Studio Code with the Go plugin.

I first tried using the vim-go plugin, but I couldn’t make it work in a custom runtimepath. I didn’t want to brick the Vim setup I have for my day-to-day work, so that’s why I tried a custom location.

Next I tried the Gogland from JetBrains, but when I ran it, my laptop sounded like a jet when the fans kicked off, so… yeah.

Then I tried Visual Studio Code, installed the Go plugin and I was pleasantly surprised when the editor figured out I was missing some Go programs like gofmt, golint, etc., and offered me to install them all. Great user experience for newcomers to Go.

About those docs…

After printing out the required “Hello, world!” to the terminal, I set out to figure out how to read in stuff from the standard input.

I would really love to say I figured it out all on my own how to do it, just by reading the official documentation, but I couldn’t. Maybe I don’t know how to use Go’s docs, maybe I’m spoiled by PHP’s excellent documentation, but after an hour or so flipping through the docs, I ended up Googling the solution.

From the documentation, I couldn’t figure out does the solution lie within the io, or the bufio package. Maybe both? I’ll have to tinker with them some more to understand the difference between the two.

From all the answers I found on the internet about this, I went with what seems the simplest way of reading from standard input: using a bufio.Scanner. Again, I’ll have to poke more around this, to see all the different ways I can accomplish this and compare the solutions.

The first “program”

The goal is to read whatever the user types in to standard input, line by line.

First, I have a slice of strings. Not an array, but a slice. I believe the main difference is that an array has a set size, while a slice’s size can change during execution.

// Inputs holds the inputs read in from stdin
type Inputs []string

I’m not yet 100% sure does this create a new type “Inputs” which is composed of a slice of strings, or is it just a slice of strings named “Inputs”? I think it’s the former, because I can use it as a return type-hint in a function.

Then in the main() I get a new scanner for the standard input:

scanner := bufio.NewScanner(os.Stdin)

It returns a pointer to the Scanner, so I pass it as such to my ReadInput function:

is := ReadInput(*scanner)

The ReadInput function takes a scanner of bufio.Scanner type and returns Inputs:

func ReadInput(scanner bufio.Scanner) Inputs

I declare two variables, one to hold the string read in from stdin, the other to “collect” all the input strings:

	var t string
	var i Inputs

Then I create an endless for loop, from which I break out when the user enters a new line only. In this loop the Scanner scans the stdin, and gets the text from it. If the text entered is not empty, I append it to the Inputs, as it is a slice of strings:

	for true {
		fmt.Print("Add a new todo [empty to stop reading]: ")
		scanner.Scan()
		t = scanner.Text()

		if err := scanner.Err(); err != nil {
			fmt.Println("Error reading from input: ", err)
		}

		// scanner.Text() strips new lines
		// so in case of just a new line
		// it's actually an empty string
		if t == "" {
			break
		}

		i = append(i, t)
	}

and finally I return the Inputs from the ReadInput function:

    return i

Back in main I do a “sanity” check by printing out the number of lines read in, and printing them out one by one:

	fmt.Printf("Got %d inputs from stdin:\n", len(is))

	for _, v := range is {
		fmt.Println(v)
	}

When executed, it looks something like this:

$ go run readstdin.go 
Hello, world!
Add a new todo [empty to stop reading]: Learn go
Add a new todo [empty to stop reading]: Write blog posts
Add a new todo [empty to stop reading]: 
Got 2 inputs from stdin:
Learn go
Write blog posts

The entire file is available here.

With this I learned a bit about Go’s types, creating functions, using the for loop, pointers, slices vs. arrays, and, of course, reading stuff from the standard input.

Happy hackin’!

P.S.: It feels good to have a beginners mind, again, about something.

Robert Basic

Robert Basic

Software engineer, consultant, open source contributor.

Let's work together!

If you require outsourcing or consulting help on your projects, I'm available!

Robert Basic © 2008 — 2018
Get the feed