Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 20 11:04AM +0930
Resurrecting an old thread.
https://groups.google.com/d/topic/golang-nuts/4aFAxBEjjBY/discussion
I'm using gob to serialise very large data sets for storage rather than
message passing (maybe
...more
|
Egon <egon...@gmail.com>: Apr 19 11:25PM -0700
On Monday, 20 April 2015 04:34:44 UTC+3, kortschak wrote:
> other approach if this is the case - it seems like the best option at
> this stage). Some cases leave me with gobs that are on the order
...more
|
Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 20 04:19PM +0930
On Sun, 2015-04-19 at 23:25 -0700, Egon wrote:
> Can you give the data-structure(s)?
> How often do you read/write the whole data?
> Do you need random access to data?
Slices of named 4-tuples
...more
|
Egon <egon...@gmail.com>: Apr 19 11:55PM -0700
On Monday, 20 April 2015 09:49:59 UTC+3, kortschak wrote:
> > How often do you read/write the whole data?
> > Do you need random access to data?
> Slices of named 4-tuples (struct of ints).
...more
|
Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 20 04:27PM +0930
On Sun, 2015-04-19 at 23:55 -0700, Egon wrote:
> Or do you mean that the slices are named?
> type Data []struct { Name string; Values []Tuple }
> Also, how long are the names?
...more
|
Sebastien Binet <seb....@gmail.com>: Apr 20 09:04AM +0200
using encoding/binary isn't that brittle and seems to have rather nice
performances (size, speed):
http://ugorji.net/blog/benchmarking-serialization-in-go
I went with encoding/binary for my disk
...more
|
Egon <egon...@gmail.com>: Apr 20 12:46AM -0700
On Monday, 20 April 2015 09:57:48 UTC+3, kortschak wrote:
> }
> type Traps []Trap
> We are keeping Traps.
Does the order matter?
First I would change the Top/Bottom ...
...more
|
Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 20 05:34PM +0930
On Mon, 2015-04-20 at 00:46 -0700, Egon wrote:
> Does the order matter?
No, not really.
> First I would change the Top/Bottom ... to int8, int16, int32, int64
> depending on what is needed.
...more
|
Egon <egon...@gmail.com>: Apr 20 01:48AM -0700
On Monday, 20 April 2015 11:04:28 UTC+3, kortschak wrote:
> > Then I would just mmap the whole structure to a file.
> This is something I'd rather avoid. This is research software and is
...more
|
Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 20 06:36PM +0930
On Mon, 2015-04-20 at 01:48 -0700, Egon wrote:
> convert from old format to a newer format?
> Or simply for experimentation it's easier if you don't have to worry about
> serialization?
...more
|
Rob Pike <r...@golang.org>: Apr 20 07:56AM -0700
The underlying issue is that the gob protocol requires that the whole
message be in memory both for encoding and decoding. An arbitrarily large
message cannot work.
I am uncomfortable just
...more
|
Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 20 06:19PM
Thanks. Comments in line.
> The underlying issue is that the gob protocol requires that the whole message be in memory both for encoding and decoding. An arbitrarily large message cannot work.
...more
|
Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 20 06:30PM
To clarify, the default limit is still tooBig (1<<30), but can be altered within this range.
...more
|
Rob Pike <r...@golang.org>: Apr 20 11:35AM -0700
The entire message is built in memory before being transmitted, so the
length can be written as a prefix.
-rob
...more
|
Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 20 06:40PM
I don't see how making the limit per-decoder harms that.
...more
|
Rob Pike <r...@golang.org>: Apr 20 12:30PM -0700
I was just rebutting your point that it is possible to write arbitrarily
large messages. You are limited by available memory as well as the static
check.
-rob
On Mon, Apr 20, 2015 at 11:40
...more
|
Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 20 08:54PM
OK, so it's not really a rebuttal then; I can write 10GB gobs right now - I can't read them. So it is already the case that for some (probably not small) set of machines it is possible to wrte gobs
...more
|
Rob Pike <r...@golang.org>: Apr 20 02:27PM -0700
I am resisting because APIs that let you choose parameters like this enable
bad designs. The constraint is there for a reason: bad data can cause bad
things to happen to the heap, and the easiest
...more
|
Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 20 09:52PM
Thanks. So are you saying that if someone were to set the limit too high for their machine to cope with the input that will lead to heap corruption?
I'm really struggling to see how this would
...more
|
Rob Pike <r...@golang.org>: Apr 20 03:23PM -0700
I am saying that I'd prefer not to change the API, which implies, as I said
before, that your best solution is to change it yourself or perhaps write a
custom codec.
I really dislike knobs in
...more
|
Dan Kortschak <dan.ko...@adelaide.edu.au>: Apr 21 07:57AM +0930
On Mon, 2015-04-20 at 15:23 -0700, Rob Pike wrote:
> I really dislike knobs in APIs.
> The code should probably complain in the encoder if it's too big; I'll
> file an issue for that.
...more
|
Ugorji Nwoke <ugo...@gmail.com>: Apr 20 04:26PM -0700
Follow up to what Rob said, you can write a custom codec or use one which
supports explicit delimiters at end of lists (like JSON "]"), as opposed to
length-prefixing. cbor is a really good
...more
|
Marvin Renich <mr...@renich.org>: Apr 20 11:01AM -0400
> > almost 8 hours without anyone offering to translate or suggesting to the
> > OP to use golang-ru instead.
> and this, ladies and gentlemen, is why we can't have nice things.
...more
|
Marvin Renich <mr...@renich.org>: Apr 20 11:02AM -0400
Sorry for the late reply. I was on vacation and then catching up on
other matters. I am trimming substantially less than I would otherwise
to give better context to a two-week-old thread.
...more
|
"atd...@gmail.com" <atd...@gmail.com>: Apr 20 09:12AM -0700
Давай! Why so serious ;)
...more
|
Joubin Houshyar <jhou...@gmail.com>: Apr 20 01:54PM -0700
حرف حساب میزنه, well
(& no, that does not mean "word count increased" ..;)
On Saturday, April 4, 2015 at 10:40:49 PM UTC-4, Robert Melton wrote:
...more
|
Benjamin Measures <saint....@gmail.com>: Apr 20 03:24PM -0700
> I was not rude, and I gave a specific reason why I believed it was inappropriate to have a conversation in a different language on the mailing list. My criticism was constructive and based on
...more
|
mm <pistac...@gmail.com>: Apr 20 09:22AM -0700
Could someone point me to a nice implementation of how CRUD is used in GO?
So far I have found this -
http://paste.ubuntu.com/8503695/
but it is incomplete. I come from Python, so I really need an
...more
|
DV <dimiter....@gmail.com>: Apr 20 10:26AM -0700
This is a strange question.
You're looking for an example that demonstrates how to do Create, Update,
Delete with a relational database in Go? You can start
...more
|
mm <pistac...@gmail.com>: Apr 20 10:41AM -0700
Sorry for being so oblique. Im using mgo/mongodb
I am trying to understand, specifically how to do GET and POST... usually
in python I have something like this:
class Edit(BaseHandler,
...more
|
Egon <egon...@gmail.com>: Apr 20 11:46AM -0700
Read http://golang.org/doc/articles/wiki/ it has an example how to use FormValue.
Then http://astaxie.gitbooks.io/build-web-application-with-golang/content/ for a thorough explanation how to build
...more
|
mm <pistac...@gmail.com>: Apr 20 12:38PM -0700
Ok found what I am looking for...Using Gocraft:
You can capture path variables like this:
router.Get("/suggestions/:suggestion_id/comments/:comment_id")
In your handler, you can access them
...more
|
mm <pistac...@gmail.com>: Apr 20 12:39PM -0700
Thanks so much Egon, those are great links for me!!
On Monday, April 20, 2015 at 2:46:36 PM UTC-4, Egon wrote:
...more
|
Guillermo Estrada <phro...@gmail.com>: Apr 20 01:51PM -0700
You're probably looking how to do an app using a framework (router at
least) and a database backend. I read you use MongoDB in Python, so for
that you should use the excellent Mongo library by
...more
|
Jason Phillips <jasonrya...@gmail.com>: Apr 19 04:30PM -0700
There's also nothing stopping you from setting GOPATH per project, like
many gophers do.
On Saturday, April 18, 2015 at 4:56:50 PM UTC-4, Thiago Farina wrote:
...more
|
andrewc...@gmail.com: Apr 19 05:17PM -0700
As other people have said, write a script which just sets it then builds.
That being said, the GOPATH variable is the reason why the rest of go works
so seamlessly.
GOPATH allows:
...more
|
Thiago Farina <tfa...@chromium.org>: Apr 20 05:21PM -0300
On Sun, Apr 19, 2015 at 8:30 PM, Jason Phillips <jasonrya...@gmail.com
> There's also nothing stopping you from setting GOPATH per project, like
> many gophers do.
> That does not seem to
...more
|
Shawn Milochik <shawn...@gmail.com>: Apr 20 04:32PM -0400
See this, it should help:
https://golang.org/doc/code.html#GOPATH
You can have multiple items in GOPATH, just as you can in PATH. And you
don't *have to* set environment variables, as others
...more
|
"John Souvestre" <jo...@sstar.com>: Apr 20 11:39AM -0500
I'm trying to get the numeric values (value property) for some non-ASCII code
points which are decimal digits (as per Unicode.IsDigit). I believe that in
Java you would use getNumericValue, and in
...more
|
Konstantin Khomoutov <flat...@users.sourceforge.net>: Apr 20 08:17PM +0300
On Mon, 20 Apr 2015 11:39:37 -0500
> getNumericValue, and in .Net GetNumericValue, for example. How can I
> do this in Go?
> Thanks,
Go uses type `rune` to represent individual Unicode code
...more
|
Konstantin Khomoutov <flat...@users.sourceforge.net>: Apr 20 08:20PM +0300
On Mon, 20 Apr 2015 11:39:37 -0500
> Unicode.IsDigit). I believe that in Java you would use
> getNumericValue, and in .Net GetNumericValue, for example. How can I
> do this in Go?
...more
|
"John Souvestre" <jo...@sstar.com>: Apr 20 12:39PM -0500
> `rune` is defined to be `int32`, so just type-convert your rune value to
`int32` to get its "value property".
I believe that you would just get the of the code point, not the value which
...more
|
Jason Gade <jay...@gmail.com>: Apr 20 10:55AM -0700
I don't think that's what he's looking
for. http://docs.oracle.com/javase/7/docs/api/java/lang/Character.html#getNumericValue(char)
If someone gives him a Chinse or Tamil or Roman numeral in
...more
|
Doug Henderson <djnd...@gmail.com>: Apr 20 12:09PM -0600
Hi,
Go does not seem to have the full unicode data that we find in the python
unicodedata module.
You may have to scan each individual rune (or []byte containing the utf-8).
...more
|
"John Souvestre" <jo...@sstar.com>: Apr 20 01:18PM -0500
Ø IIRC, Unicode.IsDigit checks for the Nd category only. I suspect that other numeric categories will not be accepted in numbers.
Yes, Unicode.IsDigit does identify decimal digits for all of the
...more
|
Andy Balholm <andyb...@gmail.com>: Apr 20 11:50AM -0700
That sounds like something that would belong in the unicode package, but apparently no one has implemented it yet.
...more
|
"John Souvestre" <jo...@sstar.com>: Apr 20 02:13PM -0500
> That sounds like something that would belong in the unicode package, but
apparently no one has implemented it yet.
Yes, that would be nice! Meanwhile, I think I'll make a map and load the 230
...more
|
"Jon Valdés" <jua...@gmail.com>: Apr 20 12:49AM -0700
Hi there,
I'm working on a project where I'm cross compiling on every build (for
linux amd64 and linux arm6, while working on a mac), and I've noticed
compilation times are much much longer
...more
|
Dave Cheney <da...@cheney.net>: Apr 20 01:20AM -0700
remember the rule, go build builds, then discards, go install installs a
copy.
the caviet to this is, dependig on where you have built go, you may
encounter a permission error.
...more
|
"Jon Valdés" <jua...@gmail.com>: Apr 20 03:28AM -0700
Hi Dave,
Not very intuitive I'd say, but that did indeed solve it.
Thanks a bunch!
On Monday, April 20, 2015 at 10:20:58 AM UTC+2, Dave Cheney wrote:
...more
|
Carlos Castillo <cook...@gmail.com>: Apr 20 08:59AM -0700
Despite Dave's usual (and correct) suggestion to use go install more often,
I think for cross-compiling (especially to multiple targets) it is less
useful. The -i flag to go build will do the
...more
|
"Jon Valdés" <jua...@gmail.com>: Apr 20 07:24PM +0200
Oh, that's a nice feature I didn't know about!
Thanks a lot, Carlos!
Kind regards,
Jon Valdes
...more
|
