Concurrency Worker Pool Implementation on Golang

Basic Definition

A Worker pool is a goroutine management technique in concurrent programming in Go-Lang with powerfull processes. A number of workers are executed and each has the same task of completing a number of jobs. With this worker pool method, the use of memory and program performance will be optimized.

Program Implementation

output: `

Send Job: 0 -> Send Job: 1 -> Send Job: 2 -> Send Job: 3 -> Send Job: 4 -> Send Job: 5 <- receive job : 0 <- receive job : 4 -> Send Job: 6 -> Send Job: 7 <- receive job : 5 -> Send Job: 8 <- receive job : 2 <- receive job : 3 -> Send Job: 9 -> Send Job: 10 <- receive job : 1 <- receive job : 9 ^Csignal: interrupt

## Discussion
In the example program above we will print a value that is * * input *. Method `echoWorker` is a function that will take the value of *input* then we make *sleep* a few *miliseconds*, suppose in the *sleep* time we seem to do another process. While the method `producer` provides input values into `channel` which is sequential.

When we run the program we must first initialize the `channel` *input* and *output*. why do we use 2 variables? So `in` is used to set the input that will be processed in the worker while `out` is used to receive the results of the *worker* process to be processed at the next stage.

The first step is to prepare the * worker* using * goroutine*, the number of workers adjusts to the needs, in the program above we use 5 workers that will run. When we execute `go producer(in)` then the `in` channel is constantly filled with continuous iterations if we do not stop the program.

We can see the output of the program, because we first set up 5 workers, then when `in` is set to data, the 5 workers run simultaneously with different processes (times). So why are the first ones received or printed are **job 0** and **job 4**? It is possible that the **Job 0** and **job 4** processes have a smaller waiting time than the others. Likewise, other workers, whoever finishes first will send the value and display the value directly.

## Conclusion
**Worker pool** is a type of *concurrent* that is easy to use to process a lot of data with a fast process. For example, if we want to move data from CSV into a *database*, and the *records* can be up to millions, this **worker pool** can be the right choice to process such data. 

With such a fast process, keep in mind that we also need to consider the number of **workers** that are running adjusted to the capacity of the computer/server where the **worker pool** will run. Do not let when we run the program on our computer/server there is a `hang` because the CPU reaches 100%.