Re: [go-nuts] Digest for golang-nuts@googlegroups.com - 8 updates in 4 topics
91 views
Skip to first unread message
Pete Wilson
Jan 18, 2021, 9:13:59 PM1/18/21
to golan...@googlegroups.com
No need to beg forgiveness.
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
http://www.bsc.es/disclaimer
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
http://www.bsc.es/disclaimer
For me, the issue is the synchronisation, not how the collection is specified.
You’re right; there’s no reason why a slice (or an array) couldn’t be used to define the collection. In fact, that’s the plan.
But the synchronisation is still a pain.
FWIW I built a channel-based experiment.
I have N worker threads plus main
I have an N channel array toworkers; each gets a pointer to its own channel
I have an N-buffered channel fromthreads
workers wait for an input on their input
when they get it, they send something on the buffered channel back to main
..each loops L times
main sends to each worker on its own channel
main waits for N responses on the communal buffered channel
.. loops L times
The end result is that
- at one goroutine per core, message send or receive costs 100-200 nsec
- but the 6-core, 12-virtual core processor only runs at 2.5 cores worth of load.
- presumably, the go runtime has too much overhead timeslicing its thread when there’s only one goroutine per thread or core
If we put 32 threads per core, so the go runtime is busily engaged in multiplexing goroutines onto a thread, cost per communication drops to around 50-60 ns
For my purposes, that’s far too many threads (gororutines) but it does suggest that synchronisation has an upper limit cost of 50 ns.
So tomorrow we try the custom sync approach. Results will be published.
Meanwhile, thanks to all for thoughts and advice
— P
On Jan 18, 2021, at 5:23 PM, golan...@googlegroups.com wrote:
- [ANN] New Go package: xmorph - 1 Update
- nil pointer ref can't figure out why - 3 Updates
- Advice, please - 2 Updates
- confusing compile error message after renaming module - 2 Updates
Scott Pakin <scot...@pakin.org>: Jan 18 12:24PM -0800
It's perhaps a bit of a niche tool, but I just released *xmorph*, a package
for warping and morphing images. The code is available from
https://github.com/spakin/xmorph
(If nothing else, you can see there my crude attempt at morphing the
cartoon Go gopher into the corresponding plush toy.)
The Go package is essentially a wrapper for the C libmorph library (
http://xmorph.sourceforge.net/), which is what the morph CLI and the xmorph
and gtkmorph GUIs build upon.
Enjoy!
— Scott
Alexander Mills <mll...@justin.tv>: Jan 18 12:00PM -0800
I have this line:
fmt.Printlin("the dao:", dao);
and it logs:
the dao: &{<nil> <nil>}
the overall method looks like:
func (dao *UserAttributeDao) GetDecryptedUserAttributes(workflowID string)
(*PayoutUserAttributeRecord, error) {
getItemInput := &dynamodb.GetItemInput{
TableName: aws.String(payoutUserAttributesTableName),
ConsistentRead: aws.Bool(true),
Key: map[string]*dynamodb.AttributeValue{
payoutUserAttributesPK: {S: aws.String(workflowID)},
},
}
fmt.Println("the dao:", dao)
record, err := dao.GetItem(getItemInput) // NIL POINTER REF HERE
if err != nil {
logrus.WithError(err).Errorf("error during GetItem from DynamoDB table %v
for id: %v",
payoutUserAttributesTableName, workflowID)
return nil, err
}
does anyone know why calling the method would result in a nil pointer? To
me it seems like the object for which the method is being called is nil,
but that doesn't make that much sense to me. My main method looks like:
func main() {
log.Println("doing some printing")
var d = new(lib.UserAttributeDao)
x, err := d.GetDecryptedUserAttributes(""); // THIS RESULTS IN NIL POINTER
if err != nil {
log.Fatal(err)
}
}
....
Axel Wagner <axel.wa...@googlemail.com>: Jan 18 09:16PM +0100
On Mon, Jan 18, 2021 at 9:12 PM 'Alexander Mills' via golang-nuts <
> does anyone know why calling the method would result in a nil pointer? To
> me it seems like the object for which the method is being called is nil
No, it is a pointer to a struct with two fields, both of which are nil. It
says `&{<nil> <nil>}`, not `<nil>`.
> var d = new(lib.UserAttributeDao)
You are initializing `d` to a pointer, pointing at the zero value of
`lib.UserAttributeDao` - which is a struct with two fields, I assume. So,
for the zero value, both of those are nil.
Axel Wagner <axel.wa...@googlemail.com>: Jan 18 09:17PM +0100
You might want to use `Printf("the dao: %+v", dao)`, for example - it will
also print type-names and should make it more obvious what you have.
On Mon, Jan 18, 2021 at 9:16 PM Axel Wagner <axel.wa...@googlemail.com>
wrote:
Kevin Chadwick <m8il...@gmail.com>: Jan 18 12:39PM
On 1/17/21 4:46 PM, Bakul Shah wrote:
> With linked lists you’re wasting half of the memory bandwidth and potentially
> the cache. Your # of elements are not going to change so a linked list doesn’t
> buy you anything. An array is ideal from a performance PoV.
Potentially forgive me and ignore this message if inappropriate as I haven't
been paying close attention to this thread at all really.
However, it strikes me that arrays are perfectly usable in GO. Similarly to how
you might use a global array with tinygo to ensure memory usage limits are not
breached. What is the issue of using an array in Go? Even a global one, *IF*
suited to the task at hand and dishing the work out to workers with a scheme a
little more complex than odd, even etc. as required?
With the benefit that an off by one etc. causes a panic and not something
potentially worse?
Bakul Shah <ba...@iitbombay.org>: Jan 18 10:50AM -0800
> breached. What is the issue of using an array in Go? Even a global one, *IF*
> suited to the task at hand and dishing the work out to workers with a scheme a
> little more complex than odd, even etc. as required?
Go arrays are perfectly usable. My comment had more to do with the fact that
you'd just have a fixed number of threads and if synchronization using channels
is not fast enough, you'd have to roll your own, in which case Go doesn't
really buy you much. I'd just keep the simulator code as a separate process
that does nothing but simulation and have it communicate with a separate
program that does visualization, logging etc. which can be in Go. Pete Wilson
in his response said that simulated cores are already generated functions in C
so this is not a big step. I'd just have the code generator generate the whole
simulator or something like it. Just different tradeoffs to consider.
DrGo <salah....@gmail.com>: Jan 17 08:14PM -0800
Wondering if someone knows what is going on here:
I renamed a module in go.mod from liverserver [sic] to gols and imported it
in main.go in a "cmd" subdir. Now I am getting this error when I compile
main.go:
./main.go:8:2: imported and not used: "github.com/drgo/gols" as liverserver
./main.go:13:12: undefined: gols
./main.go:60:12: undefined: gols
These are the lines referred to in the compiler output above
import (
"github.com/drgo/gols"
)
var config gols.Config
if err :=gols.Serve(&config); err!= nil {
go mod tidy and go go clean -cache -modcache -i -r did not make a
difference.
why the compiler still wants to import gols as liverserver
I am using go1.16beta1 darwin/amd64
Thanks,
DrGo <salah....@gmail.com>: Jan 17 08:21PM -0800
Of course, once you post about a problem, you immediately figure it out.
I forgot to update the go files in the package folder to reflect the name
change
package liverserver--> package gols fixes the issue.
On Sunday, January 17, 2021 at 10:14:36 PM UTC-6 DrGo wrote:
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page.
To unsubscribe from this group and stop receiving emails from it send an email to golang-nuts...@googlegroups.com.
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
http://www.bsc.es/disclaimer
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
http://www.bsc.es/disclaimer
Robert Engels
Jan 18, 2021, 9:59:15 PM1/18/21
to Pete Wilson, golan...@googlegroups.com
Channels are built with locks which without biased locking can lead to delays as routines are scheduled / blocked under -especially under contention.
github.com/robaho/go-concurrency-test might illuminate some other options to try.
On Jan 18, 2021, at 8:13 PM, Pete Wilson <peter....@bsc.es> wrote:
No need to beg forgiveness.
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1033748D-DA67-417C-AAC8-391409034FF2%40bsc.es.
Peter Wilson
Jan 18, 2021, 10:11:08 PM1/18/21
to golang-nuts
Robert
I was interested in channel peformance only because it's the 'go idiom'. Channel communication is relatively complex, and so is a useful upper limit to costs of inter-goroutine synchronisation.
It's reassuring to see that 50nsec per operation is what my machinery delivers. Simpler operations with atomics will therefore be no slower, and quite possibly quicker. This is good. And checkable by experiment
Once the sync costs are settled, I suspect it's the load-balancing which will rate-limit. That's more of a concern. But first things first.
Bryan C. Mills
Jan 20, 2021, 9:47:53 AM1/20/21
to golang-nuts
50ns is approximately the cost of a cross-CPU L2 cache miss. Any time you have tight cross-CPU communication, you're going to incur that cost no matter how the communication is performed — whether it's via a sync.Mutex, a channel, or an atomic write.
The way to eliminate that cost is to eliminate or batch up the cross-core communication, so that you incur less than one cross-core cache miss per operation (compare https://youtu.be/C1EtfDnsdDs).
Reply all
Reply to author
Forward
0 new messages