Networking in Go

[Kanak Bhatia]

I want to learn networking in Go. What are the best resources to refer to and projects to practice? Would love some guidance.

[Jason E. Aten]

Hi Kanak. You'll get better answers with a little bit longer description of what specifically you want to learn. Just "networking" is too broad a topic to offer any concise guidance--its like saying "I want to know about 'life', or 'the earth', or 'the internet'". What problems do you face? What things about networking to you find challenging, opaque, bewildering?  What do you know already/what is your background so far? 

Best wishes,

Jason

[Zhang Jie (Kn)]

If you want to learn network programming in Go, you can check this project [trpc](https://github.com/Tencent/trpc).

trpc-cpp, trpc-go, trpc-java and other haven't been opensourced language versions.

This is a microservice framework built for Tencent. You can learn many network programming best practices from it.

Maybe you think it's difficult to start from a framework. It's not, because we add many readable and maintainable tests for nearly every feature.

The code testing coverage goes to 80%+.

You can read the code and run the tests to learn as you need without even build a project based on it.
If you have experience in cpp, java, you can also check trpc-cpp, trpc-java to compare with the trpc-go version.

Hope this can help you.

[alex-coder]

Hi,

there are 4 books. it is possible that there are out of dates a little bit:

1. Network Programming with Go by Jan Newmarch.  This book is hosted on github-pages

2. NETWORK PROGRAMMING WITH GO. © 2021 by Adam Woodbeck

3. Network Programming with Go Language: Essential Skills for Programming, Using and Securing Networks with Open Source Google Golang  Copyright © 2022 by Jan Newmarch and Ronald Petty

4. Network Automation with Go,  Copyright © 2022 Packt Publishing

Regards.

вторник, 11 марта 2025 г. в 14:57:16 UTC+3, Zhang Jie (Kn):

[Kanak Bhatia]

Thanks for your response! I wanted to clarify my focus.

I recently started exploring MinIO, an object storage system written in Go, and realized that I need a better understanding of networking protocols like TCP and UDP in the context of storage systems. MinIO uses S3-compatible APIs, encryption, and distributed architecture, which got me thinking about how such systems handle networking and security.

I’d like to understand:

1. Networking in Object Storage – How TCP/UDP, HTTP APIs, and data transfer mechanisms work in systems like MinIO.

2. File to Object Conversion – How files are stored as objects and retrieved efficiently.

3. Encryption & Security – How data is encrypted both in transit and at rest.

4. Backup & Storage Concepts – Key networking and storage fundamentals relevant to cloud storage.

5. How authentication, load balancing, and distributed networking are handled in Go.

Could you recommend structured resources or approaches to learning these topics?

I have around 2 years experience in the Java Springboot Domain, work mostly concerned around web development making APIs.

My main issue is that I'm looking at an open-source project in Go and trying to understand its flow, along with the different networking terminologies used in it. Specifically, while going through MinIO's code, I’ve come across terms like gRPC streaming, HTTP multipart uploads, connection pooling, TLS handshakes, and distributed erasure coding. Understanding how these pieces fit together, especially in the context of high-performance storage systems, is where I’m struggling.

[Jason E. Aten]

Hi Kanak,

Thanks for giving us more detail.  This is a fascinating

and incredibly useful problem space (networking and storage).

It is well worth diving into; sub-spaces in it become

people's whole careers. 

As you've observed, a finished product like MinIO will use alot of technology.

The thing to do is to take a divide and conquer approach, and to just

focus on one part at a time. Recursively split that part into its

components until you have something you can tackle in a reasonable

time span -- whatever that is for your available time. 

 And don't feel rushed.  Computers run so fast that they hide a 

very large amount of complexity--and can deceive us into thinking

that these things "must be simple". These are complex topics that will

take quite some time to comprehend--especially if you have to come

up to speed with the math behind them--and even quite a bit of time 

if you ignore the math, which is sometimes viable. Understanding the

TLS handshake, for example, from scratch, probably took me a good 3-6 months. 

One trick is to choose a problem to focus on. By taking

a problem focus, you can turn learning into a fun game. Pick a 

problem that you don't understand the answer to (yet), but that

you know *has* an answer, because others have solved it.

By dividing and conquering, and focusing on specific problems, 

you avoid being overwhelmed by too much at once.

Then the thing that I find most help in learning a technology is to take

a hands on approach, and to try and "solve the problem myself" first, or at least

some small version of the problem in the space that I want to understand. 

That is, actually write code to solve, or try to solve, the problem. 

By trying to tackle a small, focused problem yourself, without "looking at the answers" first,

you come to an intuitive understanding of the problem, and thus

the design space solutions must take. 

You'll either find an easy solution nobody else thought of, or you'll

run into a dead end and then truly appreciate the cleverness

of the solution when you do "turn to the back of the book" to

check your answer.  Because that's the next step--once you

have come to grips with the problem space, then you can look at the

answers that other people have done. But still keep "hands-on"

as you do so. Take the clever ideas that you encounter in

looking at "the answers", and incorporate them into your

own solution code.

To give a specific example for networking, the area

you are trying to tackle, I found it most helpful to come to grips

with TCP by implementing the sliding window protocol

that TCP uses for reliable stream transmission.

The problem is stated thusly: how can I turn an unreliable, packet based

communication system into a reliable, infinite stream of bytes

between two hosts?  I would minimize this at first to focus

on the essentials: how can you reliably get a  1000 byte blob from one

goroutine to another goroutine, when you must send it in

parts through a go channel of type "chan [10]byte", *and* you are 

forced to discard 20% of your receives at random on the receiving

end to simulate a lossy wifi network.

Good luck! Enjoy the learning process.

Best wishes,

Jason

[Kanak Bhatia]

Thank you all for the suggestions!! I will look into them and will start focusing on one problem at a time.