My father-in-law is a mostly retired mining engineer. He keeps a lot of data about mines in an Excel spreadsheet.
I'm a QA tester. I've been working on a data-driven Fit-like harness that holds test data in Excel spreadsheets.
He had to do some math on the values in the table. I hacked up my test data framework to parse his spreadsheet (took about 15 minutes), and now he's learning Ruby (and the very nice ScriptingExcel page at Ruby Garden) because he just couldn't do his work with VBA.
Chris McMahon wrote: > He had to do some math on the values in the table. I hacked up my test > data framework to parse his spreadsheet (took about 15 minutes), and > now he's learning Ruby (and the very nice ScriptingExcel page at Ruby > Garden) because he just couldn't do his work with VBA.
I too am learning Ruby (I am a mere 30 though) and have started with hacking at Excel spreadsheets as a) I have to do a lot of this in my work, thus making rather dry work quite fun (ah, the magic of Ruby); and b) VBA makes my skin crawl. The Scripting Excel page on the Garden has been invaluable for this, but I'm finding reading in values from cells and writing them back again to be extremely slow. Once in a Ruby array they process like lightening. Has anyone got any tips for speeding up the parsing of ranges, and writing them back again?
Here's an example of how I'm iterating over a range (simplified for clarity):
On large ranges, where I slot arrays within arrays, doing this kind of thing seems to take an age, even on my Athlon64 3200. To give you some idea, reading in an 8000 row x 20 column worksheet takes over 1 minute and 45 seconds, while writing takes about 80 seconds. The actual processing Ruby does on the arrays is about 1 second from what I can tell.
Chris - I'd love to take a look at your Excel parsing framework. Perhaps it'll help me get my head around Ruby a little more.
> Chris McMahon wrote: > > He had to do some math on the values in the table. I hacked up my test > > data framework to parse his spreadsheet (took about 15 minutes), and > > now he's learning Ruby (and the very nice ScriptingExcel page at Ruby > > Garden) because he just couldn't do his work with VBA.
> I too am learning Ruby (I am a mere 30 though) and have started with > hacking at Excel spreadsheets as a) I have to do a lot of this in my > work, thus making rather dry work quite fun (ah, the magic of Ruby); > and b) VBA makes my skin crawl. The Scripting Excel page on the Garden > has been invaluable for this, but I'm finding reading in values from > cells and writing them back again to be extremely slow. Once in a Ruby > array they process like lightening. Has anyone got any tips for > speeding up the parsing of ranges, and writing them back again?
> Here's an example of how I'm iterating over a range (simplified for > clarity):
> On large ranges, where I slot arrays within arrays, doing this kind of > thing seems to take an age, even on my Athlon64 3200. To give you some > idea, reading in an 8000 row x 20 column worksheet takes over 1 minute > and 45 seconds, while writing takes about 80 seconds. The actual > processing Ruby does on the arrays is about 1 second from what I can > tell.
> Chris - I'd love to take a look at your Excel parsing framework. > Perhaps it'll help me get my head around Ruby a little more.
On 10/27/05, Charles Roper <charles.ro...@gmail.com> wrote:
> The Scripting Excel page on the Garden > has been invaluable for this, but I'm finding reading in values from > cells and writing them back again to be extremely slow. Once in a Ruby > array they process like lightening. Has anyone got any tips for > speeding up the parsing of ranges, and writing them back again?
Charles,
It's a simple trick but in case you don't know about it, turn off screen updating while you're doing your routine using: excel.screenupdating = false .where excel is your reference to the application (e.g. WIN32OLE::new('excel/application'))
Remember to turn it back on with '= true' when you're done or excel won't show your changes.
On 10/27/05, Jacob Quinn Shenker <jqshen...@gmail.com> wrote:
> On 10/27/05, pat eyler <pat.ey...@gmail.com> wrote: > > On 10/27/05, Jacob Quinn Shenker <jqshen...@gmail.com> wrote: > > > How about youngest Ruby hacker?
> > Well, my 11 yo son is working on it. I'm looking forward to helping > > him post some of his work here in the not too distant future.
> ...well, he has me beat. I'm 14. :)
He may beat you ... let's wait 'till he actually writes something more than the 'how many seconds old are you?' exercises.
That ain't nothing! ( ;) ) I was already programming when I was 6, and it wasn't in Ruby only because Ruby didn't exist yet. Hey, I had already written a fully fledged text editor when I was 14, all in BASIC (Amstrad CPC BASIC). I remember having to deal with the cursor issue and the editor buffer and the text display all by myself. Can't remember how I did it though (and I don't have the program anymore :( ).
And then one wonders why I didn't take a CS education. Must be all that brain-washing from my parents and my teachers that programming "wouldn't put bread on the table." Now I have to content myself with a M.Sc in Physics :( ...
Seriously though, it's indeed pretty cool to see people of the age of 14 already hacking away :) . -- Christophe Grandsire.
Christophe Grandsire wrote: > Selon pat eyler <pat.ey...@gmail.com>:
>>Hacking Ruby at 14 is pretty cool though. :)
> That ain't nothing! ( ;) ) I was already programming when I was 6, and it wasn't > in Ruby only because Ruby didn't exist yet. Hey, I had already written a fully > fledged text editor when I was 14, all in BASIC (Amstrad CPC BASIC).
>> That ain't nothing! ( ;) ) I was already programming when I was 6, and >> it wasn't >> in Ruby only because Ruby didn't exist yet. Hey, I had already written >> a fully >> fledged text editor when I was 14, all in BASIC (Amstrad CPC BASIC).
> Not to be snide, but "Do you want a cookie?"
I am not a native English speaker, so I don't know what you mean.
As for what I wrote, it was meant as a semi-joke (note the winkey!), just to point out in (hopefully) a humorous way that some have begun programming quite early in their life. I realise now it kind of made me look like a grumpy old man who didn't feel he was recognised enough. That wasn't my purpose (hey, I'm only 29, with the social skills of closed oyster, so I may sound quite differently from what I mean).
Anyway, if my remarks were in any way out of place, I apologise. I wasn't meaning to sound pretentious. Apparently I failed :( . -- Christophe Grandsire.
>>> That ain't nothing! ( ;) ) I was already programming when I was 6, >>> and it wasn't >>> in Ruby only because Ruby didn't exist yet. Hey, I had already >>> written a fully >>> fledged text editor when I was 14, all in BASIC (Amstrad CPC BASIC).
>> Not to be snide, but "Do you want a cookie?"
> I am not a native English speaker, so I don't know what you mean.
Growing up as a minister's son I would often go to bible camps in the summer. All of us pk's (pastor's kids) would always feel we had a one up on all of the other campers for some reason. We felt we were "special" just for being a pk! One year a speaker at a conference asked the pastor's kids if they wanted a cookie because they were special, then he told us to sit down and he went on with the rest of his message.
So the cookie comment could be rephrased by saying, "Did you tell us this just so we would all know how special you are? Would you like a prize (a cookie) for your accomplishments as a kid?)
> As for what I wrote, it was meant as a semi-joke (note the winkey!), > just to point out in (hopefully) a humorous way that some have begun > programming quite early in their life. I realise now it kind of made me > look like a grumpy old man who didn't feel he was recognised enough. > That wasn't my purpose (hey, I'm only 29, with the social skills of > closed oyster, so I may sound quite differently from what I mean).
> Anyway, if my remarks were in any way out of place, I apologise. I > wasn't meaning to sound pretentious. Apparently I failed :( .
I see what you mean, I was being way to picky. I apologize for insensitive comment on my behalf. Have a good weekend,
On 28/10/05, Terje Tjervaag <fleckt...@gmail.com> wrote:
> It's a simple trick but in case you don't know about it, turn off > screen updating while you're doing your routine using: > excel.screenupdating = false > ..where excel is your reference to the application (e.g. > WIN32OLE::new('excel/application'))
> Remember to turn it back on with '= true' when you're done or excel > won't show your changes.
Terje,
Many thanks for the suggestion. I'm actually doing something similar and hiding the whole Excel app:
excel.Visible = false
I also tried your suggestion, but reading and writing to cells was still slow.
However, I found a solution. Iterating over a range of cells and reading or writing to them individually is what was causing the bottleneck, so I experimented with reading whole ranges into an array like so:
Not only is this more terse, it's also much, much faster. Using my 8000 row sheet, it reads the range into an array in about a 1.5 seconds, as opposed to the iterator taking over 1 minute and 45 seconds.
I was also delighted to discover that this works for writing back to a range of cells too: