Concurrency with golang - how to handle background tasks

Learn how golang can handle in an easy way the background tasks with goroutine, waitgroups, or channels, and how you can pass the data to manage the concurrent tasks. You will also see some easy examples about concurrency.

Golang coroutine

As written by the golang documentation, a goroutine is a lightweight thread managed by the Go runtime. A goroutine can be started by using the go prefix.

goroutine.go
go func() {
   your asynchronous code here
}()

Every part of code that is written after the go routine, will be executed after or before the goroutine.

In order to be sure that the function written after the goroutine, will be executed after the asynchronous code, you have to use a WaitGroup, in this way.

wgProcessing.go
var wgProcessing sync.WaitGroup

wgProcessing.Add(1)
go func() {
   defer wgProcessing.Done()
   your asynchronous code here
}()

wgProcessing.Add(1)
go func() {
   defer wgProcessing.Done()
   your other asynchronous code here
}()


wgProcessing.Wait()

//executed when all goroutine are done, so if they will not end, this part of code will never be reached
fmt.Println("done")

For every goroutine, you have to add one to the wgProcessing, inside the goroutine you have to tell when the asynchronous function is done and the Wait function will wait that the waitgroup counter is zero.

Golang concurrency with channels

Let's go further with the golang channels.

Channels are a way to send and get data. They are identified by the <- operator.

In this example we will set a concurrency of 20, the input is an integer and the output is just the input transformed in string with the word output as a prefix.

channels.go
func doWork() {
	const concurrency = 20

	numberOfJobs := 2000

	jobs := make(chan int, numberOfJobs)
	results := make(chan string, numberOfJobs)
	defer close(jobs)
	defer close(results)

	for i := 0; i < concurrency; i++ {
		go worker(jobs, results)
	}

	for i := 0; i < numberOfJobs; i++ {
		jobs <- i
	}

	for i := 0; i < numberOfJobs; i++ {
		result := <-results
		fmt.Println(i, result)
	}

}

func worker(input <-chan int, output chan<- string) {
	for row := range input {
		output <- fmt.Sprintf("output: %d", row)
	}
}

Remember that in this way you don't have the control on the order of execution, so the asynchronous functions can return the value in a random way so be sure to store in the results a reference for every executed asynchronous job.

Semaphore

Now you are able to run and manage background tasks with golang, but what if you have to handle some shared variables? Usually shared variables should be accessed and written only once per time, so you have to handle when the variable is used and when is free, you can use semaphores.

semaphore.go
var semaphore = semaphore.NewWeighted(int64(1))

_ = semaphore .Acquire(session.Context, 1)
defer semaphore .Release(1)

//do the work with your shared variable

In this way your variable can be readed and written only once per time.

Multiple read, one write, RWMutex is the solution

If you allow to perform multiple read operation on a variable at the same time, but only one write per time, then RWMutex is the solution for this problem.

  • Lock(): only one go routine read/write at a time by acquiring the lock.
  • RLock(): multiple go routine can read (but not write) at a time by acquiring the lock.

RWMutex.go
var (
 Locker = sync.RWMutex{}
)


locker.Locker.Lock()
//write operation for the variable
locker.Locker.Unlock()


locker.Locker.RLock()
//read the variable
locker.Locker.RUnlock()

Other articles