Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

How to set the compiler to handle thousand loops ?

6 views
Skip to first unread message

tobleron

unread,
Oct 31, 2008, 4:41:25 AM10/31/08
to
Hi,

Hi, I'm using NetBeans IDE 6.1 to develope my application. My
application works well when deals with loops of reading 52,000 lines
of waveform data. But it fails when reads 104,000 of waveform data.
How to set the IDE to handle multiple loops like this ? I'm using
FileOutputStream function, while-do, and ByteBuffer function in this
case. Please advise. Thank you in advance.

Roedy Green

unread,
Oct 31, 2008, 5:45:07 AM10/31/08
to
On Fri, 31 Oct 2008 01:41:25 -0700 (PDT), tobleron <bud...@yahoo.com>
wrote, quoted or indirectly quoted someone who said :

see http://mindprod.com/applet/fileio.html

for to generate code for all manner of ways of reading your data.

You also might consider CSV format.
see http://mindprod.com/jgloss/csv.html
--
Roedy Green Canadian Mind Products
http://mindprod.com
A vote for McCain is fearful clinging to McSame.
A vote for Obama is a shot at Obamalot.

RedGrittyBrick

unread,
Oct 31, 2008, 5:58:06 AM10/31/08
to

Please read http://www.catb.org/~esr/faqs/smart-questions.html

- Define "it fails".
- Where is your code?
- What makes you think an IDE places any restriction on loop size?

--
RGB

Lew

unread,
Oct 31, 2008, 6:38:46 AM10/31/08
to
tobleron wrote:
>> Hi,
>> Hi,
>> I'm using NetBeans IDE 6.1 to develope my application. My
>> application works well when deals with loops of reading 52,000 lines
>> of waveform data. But it fails when reads 104,000 of waveform data.
>> How to set the IDE to handle multiple loops like this ? I'm using
>> FileOutputStream function, while-do, and ByteBuffer function in this
>> case. Please advise. Thank you in advance.

RedGrittyBrick wrote:
> Please read http://www.catb.org/~esr/faqs/smart-questions.html
>
> - Define "it fails".
> - Where is your code?
> - What makes you think an IDE places any restriction on loop size?

Hint to the OP: The IDE has nothing to do with the problem.

Check out, read thoroughly, and follow the advice of <http://sscce.org/>.

--
Lew

tobleron

unread,
Oct 31, 2008, 7:24:01 AM10/31/08
to
On Oct 31, 5:58 pm, RedGrittyBrick <RedGrittyBr...@spamweary.invalid>
wrote:

> tobleron wrote:
> > Hi,
>
> > Hi, I'm using NetBeans IDE 6.1 to develope my application. My
> > application works well when deals with loops of reading 52,000 lines
> > of waveform data. But it fails when reads 104,000 of waveform data.
> > How to set the IDE to handle multiple loops like this ? I'm using
> > FileOutputStream function, while-do, and ByteBuffer function in this
> > case. Please advise. Thank you in advance.
>
> Please readhttp://www.catb.org/~esr/faqs/smart-questions.html

>
> - Define "it fails".
> - Where is your code?
> - What makes you think an IDE places any restriction on loop size?
>
> --
> RGB

@All,

Here is the the error code shows by NetBeans IDE :

Caused by: java.lang.OutOfMemoryError: Java heap space
at ....

For mor detail, my code is to generating DICOM file which is contains
waveform data of ECG record. The waveform data source is a text file
that contains hundreds or even thousands lines of ECG signal. Each
line contains 12 ECG channels. So what my code do is :

- first : opens the text file BufferedReader function.
- second : uses FileOutputStream function to write the streamed bytes.
- third : reads line by line (by using scanner and loop function) then
split each line to get the 12 channels value.
- forth : switch to little endian encoding to write the channel value
bytes (DICOM using little endian encoding for the waveform value).

The third and forth steps are looped from the first line until the end
line of the text file.

I said : it fails, because the process is stopped and the IDE shows
error message above.

I thought the IDE can not handle thousands times of looping, because
the code works well in 52,000 lines data, but when I doubled the
lines, the process is stopped and the error message is occured. And it
was "out of memory".

So, I want to know, is there any way to make adjustment in IDE, may be
for buffer setting, or some thing like that ? I used 512 MB of RAM,
and I seek another possible ways other than "add more RAMs". Please
advise.

Lew

unread,
Oct 31, 2008, 8:10:29 AM10/31/08
to
tobleron wrote:
> On Oct 31, 5:58 pm, RedGrittyBrick <RedGrittyBr...@spamweary.invalid>
> wrote:
>> tobleron wrote:
>>> Hi,
>>> Hi, I'm using NetBeans IDE 6.1 to develope my application. My
>>> application works well when deals with loops of reading 52,000 lines
>>> of waveform data. But it fails when reads 104,000 of waveform data.
>>> How to set the IDE to handle multiple loops like this ? I'm using
>>> FileOutputStream function, while-do, and ByteBuffer function in this
>>> case. Please advise. Thank you in advance.
>> Please readhttp://www.catb.org/~esr/faqs/smart-questions.html
>>
>> - Define "it fails".
>> - Where is your code?
>> - What makes you think an IDE places any restriction on loop size?
>>
>> --
>> RGB
>
> @All,
>
> Here is the the error code shows by NetBeans IDE :
>
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at ....

You would see the same error even running outside the IDE.

Just out of curiosity, why aren't you quoting the entire error message?

OOME means you haven't allocated enough memory for the process, or else you
are wasting memory in your code (my bet). Allocate more memory or else fix
the bug.

> For mor detail, my code is to generating DICOM file which is contains

For more detail:


> Check out, read thoroughly, and follow the advice of <http://sscce.org/>.

Why the heck do you ignore advice, then ask for more help?

> waveform data of ECG record. The waveform data source is a text file
> that contains hundreds or even thousands lines of ECG signal. Each
> line contains 12 ECG channels. So what my code do is :

SSCCE, please. Description is useless.

> - first : opens the text file BufferedReader function.
> - second : uses FileOutputStream function to write the streamed bytes.
> - third : reads line by line (by using scanner and loop function) then
> split each line to get the 12 channels value.
> - forth : switch to little endian encoding to write the channel value
> bytes (DICOM using little endian encoding for the waveform value).
>
> The third and forth steps are looped from the first line until the end
> line of the text file.

Gee, if we could only see the code we could tell you where your mistake is.

> I said : it fails, because the process is stopped and the IDE shows
> error message above.

You said too little to let us help.

> I thought the IDE can not handle thousands times of looping, because

Your problem has nothing to do with the IDE.

> the code works well in 52,000 lines data, but when I doubled the
> lines, the process is stopped and the error message is occured. And it
> was "out of memory".

You are not releasing memory after each loop.

> So, I want to know, is there any way to make adjustment in IDE, may be

Nope.

> for buffer setting, or some thing like that ? I used 512 MB of RAM,
> and I seek another possible ways other than "add more RAMs". Please
> advise.

You got advice that you ignored.
<http://sscce.org/>

Generally speaking, your code is allocating memory that it never releases.
Release unused objects and they will be collected, and you won't get the OOME.

<http://sscce.org/>
<http://sscce.org/>
<http://sscce.org/>

--
Lew

Martin Gregorie

unread,
Oct 31, 2008, 8:13:47 AM10/31/08
to
On Fri, 31 Oct 2008 04:24:01 -0700, tobleron wrote:

> So, I want to know, is there any way to make adjustment in IDE, may be
> for buffer setting, or some thing like that ? I used 512 MB of RAM, and
> I seek another possible ways other than "add more RAMs". Please advise.
>

So, if I understand your description, what you're doing is:

read file into a buffer
for each line in the buffer
split into channels
re-code into DICOM (little endian) format
append line to output buffer
end-for
write output buffer to a file

If I'm wrong, please describe your logic as an SSCE or in pseudocode.

Otherwise, why buffer entire files when you appear to be processing the
data line by line? Read from the file line by line and output each
converted line as its completed and your memory problems will vanish.

--
martin@ | Martin Gregorie
gregorie. | Essex, UK
org |

RedGrittyBrick

unread,
Oct 31, 2008, 11:45:38 AM10/31/08
to

tobleron wrote:
> On Oct 31, 5:58 pm, RedGrittyBrick <RedGrittyBr...@spamweary.invalid>
> wrote:
>> tobleron wrote:
>>> Hi,
>>> Hi, I'm using NetBeans IDE 6.1 to develope my application. My
>>> application works well when deals with loops of reading 52,000 lines
>>> of waveform data. But it fails when reads 104,000 of waveform data.
>>> How to set the IDE to handle multiple loops like this ? I'm using
>>> FileOutputStream function, while-do, and ByteBuffer function in this
>>> case. Please advise. Thank you in advance.
>> Please readhttp://www.catb.org/~esr/faqs/smart-questions.html
>>
>> - Define "it fails".
>> - Where is your code?
>> - What makes you think an IDE places any restriction on loop size?
>>
>
> @All,
>
> Here is the the error code shows by NetBeans IDE :
>
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at ....
>
> For mor detail, my code is to generating DICOM file which is contains
> waveform data of ECG record. The waveform data source is a text file
> that contains hundreds or even thousands lines of ECG signal. Each
> line contains 12 ECG channels. So what my code do is :
>
> - first : opens the text file BufferedReader function.
> - second : uses FileOutputStream function to write the streamed bytes.
> - third : reads line by line (by using scanner and loop function) then
> split each line to get the 12 channels value.
> - forth : switch to little endian encoding to write the channel value
> bytes (DICOM using little endian encoding for the waveform value).
>
> The third and forth steps are looped from the first line until the end
> line of the text file.
>
> I said : it fails, because the process is stopped and the IDE shows
> error message above.
>
> I thought the IDE can not handle thousands times of looping, because
> the code works well in 52,000 lines data, but when I doubled the
> lines, the process is stopped and the error message is occured. And it
> was "out of memory".
>
> So, I want to know, is there any way to make adjustment in IDE,

You don't need to.

> may be
> for buffer setting, or some thing like that ? I used 512 MB of RAM,
> and I seek another possible ways other than "add more RAMs".

You probably need much LESS RAM if you fix your coding.

> Please advise.

See Martin Gregorie's advice. With more or less the *same* *small*
amount of memory usage you should be able to process 1 line, 1000 lines,
1000000 lines, 1000000000 lines or almost any number of lines (each
containing twelve channel values).

Just make sure your FileOutputStream is being flushed periodically and
that you are not "packratting" objects
(http://mindprod.com/jgloss/packratting.html)


--
RGB

tobleron

unread,
Oct 31, 2008, 11:47:57 PM10/31/08
to
@All,

You can download my source code at http://www.artikelilmiah.com/NewECGRecord.zip
The waveform data can be downloaded at http://www.artikelilmiah.com/12ecgsample2-104000.zip
and http://www.artikelilmiah.com/12ecgsample2-52000.zip

Here is the complete error message :

init:
deps-jar:
compile:
run:
Nov 1, 2008 11:25:26 AM org.jdesktop.application.LocalStorage getId
WARNING: unspecified resource Application.id using Main
Nov 1, 2008 11:25:26 AM org.jdesktop.application.LocalStorage getId
WARNING: unspecified resource Application.vendorId using
UnknownApplicationVendor
Exception in thread "AWT-EventQueue-0" java.lang.Error:
java.lang.reflect.InvocationTargetException
at
org.jdesktop.application.ApplicationAction.actionFailed(ApplicationAction.java:
859)
at
org.jdesktop.application.ApplicationAction.noProxyActionPerformed(ApplicationAction.java:
665)
at
org.jdesktop.application.ApplicationAction.actionPerformed(ApplicationAction.java:
698)
at
javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:
1995)
at javax.swing.AbstractButton
$Handler.actionPerformed(AbstractButton.java:2318)
at
javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:
387)
at
javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at
javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:
236)
at
java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:
272)
at java.awt.Component.processMouseEvent(Component.java:6041)
at javax.swing.JComponent.processMouseEvent(JComponent.java:
3265)
at java.awt.Component.processEvent(Component.java:5806)
at java.awt.Container.processEvent(Container.java:2058)
at java.awt.Component.dispatchEventImpl(Component.java:4413)
at java.awt.Container.dispatchEventImpl(Container.java:2116)
at java.awt.Component.dispatchEvent(Component.java:4243)
at
java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4322)
at
java.awt.LightweightDispatcher.processMouseEvent(Container.java:3986)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:
3916)
at java.awt.Container.dispatchEventImpl(Container.java:2102)
at java.awt.Window.dispatchEventImpl(Window.java:2440)
at java.awt.Component.dispatchEvent(Component.java:4243)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:
273)
at
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:
183)
at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:
173)
at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:168)
at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:160)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:
121)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jdesktop.application.ApplicationAction.noProxyActionPerformed(ApplicationAction.java:
662)
... 27 more


Caused by: java.lang.OutOfMemoryError: Java heap space

at java.util.Arrays.copyOfRange(Arrays.java:3209)
at java.lang.String.<init>(String.java:216)
at java.lang.StringBuffer.toString(StringBuffer.java:585)
at ecgterminal3.NewECGRecord.okGenerate(NewECGRecord.java:
2429)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jdesktop.application.ApplicationAction.noProxyActionPerformed(ApplicationAction.java:
662)
at
org.jdesktop.application.ApplicationAction.actionPerformed(ApplicationAction.java:
698)
at
javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:
1995)
at javax.swing.AbstractButton
$Handler.actionPerformed(AbstractButton.java:2318)
at
javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:
387)
at
javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at
javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:
236)
at
java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:
272)
at java.awt.Component.processMouseEvent(Component.java:6041)
at javax.swing.JComponent.processMouseEvent(JComponent.java:
3265)
at java.awt.Component.processEvent(Component.java:5806)
at java.awt.Container.processEvent(Container.java:2058)
at java.awt.Component.dispatchEventImpl(Component.java:4413)
at java.awt.Container.dispatchEventImpl(Container.java:2116)
at java.awt.Component.dispatchEvent(Component.java:4243)
at
java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4322)
at
java.awt.LightweightDispatcher.processMouseEvent(Container.java:3986)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:
3916)
at java.awt.Container.dispatchEventImpl(Container.java:2102)
at java.awt.Window.dispatchEventImpl(Window.java:2440)
at java.awt.Component.dispatchEvent(Component.java:4243)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:
273)
at
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:
183)
BUILD SUCCESSFUL (total time: 1 minute 58 seconds)

So, please help. As I mentioned before, it works well on 52000 lines
of waveform data (and under), but fails when trying to read 104000
lines of waveform data.

Lew

unread,
Nov 1, 2008, 12:21:53 AM11/1/08
to
tobleron wrote:
> You can download my source code at http://www.artikelilmiah.com/NewECGRecord.zip

<http://sscce.org/>

> So, please help. As I mentioned before, it works well on 52000 lines
> of waveform data (and under), but fails when trying to read 104000
> lines of waveform data.

<http://sscce.org/>

--
Lew

Lew

unread,
Nov 1, 2008, 12:43:43 AM11/1/08
to
Looks like somehow my previous message double-posted. Sorry. Not sure how
that happened.

tobleron wrote:
> You can download my source code at http://www.artikelilmiah.com/NewECGRecord.zip

This download gave me a .class file, not source.

--
Lew

tobleron

unread,
Nov 1, 2008, 8:11:51 AM11/1/08
to
On Nov 1, 12:43 pm, Lew <no...@lewscanon.com> wrote:
> Looks like somehow my previous message double-posted.  Sorry.  Not sure how
> that happened.
>
> tobleron wrote:
> > You can download my source code athttp://www.artikelilmiah.com/NewECGRecord.zip

>
> This download gave me a .class file, not source.
>
> --
> Lew

@Lew

I've tried to describe my problem as clear as I can. What kind of
<http://sscce.org/> that you want ? If you see in that .class file,
you'll know that I just put my code that I mentioned before in
okGenerate() function.

Donkey Hottie

unread,
Nov 1, 2008, 9:07:25 AM11/1/08
to
tobleron <bud...@yahoo.com> wrote in
news:f274cb57-86de-4545...@a29g2000pra.googlegroups.com:

> @All,
>
> You can download my source code at
> http://www.artikelilmiah.com/NewECGRecord.zip

That zip contains nothing more than the compiled class file. Useless.

Lew

unread,
Nov 1, 2008, 9:35:49 AM11/1/08
to
tobleron wrote:
>>> You can download my source code athttp://www.artikelilmiah.com/NewECGRecord.zip

Lew wrote:
>> This download gave me a .class file, not source.

tobleron wrote:
> I've tried to describe my problem as clear as I can. What kind of
> <http://sscce.org/> that you want ? If you see in that .class file,
> you'll know that I just put my code that I mentioned before in
> okGenerate() function.

It is very inconvenient to decompile a .class file. Why not provide source?

--
Lew

Lew

unread,
Nov 1, 2008, 9:38:07 AM11/1/08
to

The OP was already told that. Their response:


> If you see in that .class file,
> you'll know that I just put my code that I mentioned before in
> okGenerate() function.

With these reminders, perhaps they'll consider providing source.

--
Lew

Mark Space

unread,
Nov 1, 2008, 12:09:53 PM11/1/08
to
tobleron wrote:

> I've tried to describe my problem as clear as I can. What kind of
> <http://sscce.org/> that you want ?

The short, self-contained compilable kind. One class, provide "input"
for the file read so we don't need some sort of external file, try to
get the file down to 60 lines or less. It should be possible for you.

Also make sure your error (OutOfMemory) occurs in your example before
posting it.

> If you see in that .class file,
> you'll know that I just put my code that I mentioned before in
> okGenerate() function.

No one wants to read through 3000 lines of source and a dozen classes to
debug your problem. You are asking for help, you do the work to make it
obvious to us what is going on. Post a ->"short"<- example, not your
entire code base.

Arne Vajhøj

unread,
Nov 1, 2008, 2:03:56 PM11/1/08
to

Not useless.

But it is asking people that want to help waste time
decompiling.

Very few will do that.

Arne

tobleron

unread,
Nov 2, 2008, 8:54:45 AM11/2/08
to
@all,

I don't understand what you mean with compiled class file. I sent the
zip file that contains the original code. Some parts are written by
the NetBeans, but for the action function, I wrote it by hand. I don't
understand why you said that I need you all to decompile it ? I don't
need you to decompile it.

First time, I avoid to send the code because I know, it will bothering
you with that complicated code. So I tried to just describe the flow.
But you need the original code, so I sent it. But now you said I sent
too much. Another person said I sent too less. I really don't
understand what should I provide to make you understand how to solve
this looping problem.

Anyway, I tried to provide just a part of the code that been my
concern problem, which is the looping part. You can see it at
http://www.artikelilmiah.com/looping.zip I hope you can understand
what I'm concerning and can give me a better solution.

@Martin Gregorie
Hmm... I think I did just like your suggestion. Or maybe it because of
the size of the waveform data source (approx 13 MB) ?

Lew

unread,
Nov 2, 2008, 9:02:27 AM11/2/08
to
tobleron wrote:
> I don't understand what you mean with compiled class file.

The zip file contained a .class file comprising byte code. It contained no
text file at all, much less source code.

--
Lew

Joshua Cranmer

unread,
Nov 2, 2008, 9:18:37 AM11/2/08
to
tobleron wrote:
> @all,
>
> I don't understand what you mean with compiled class file. I sent the
> zip file that contains the original code. Some parts are written by
> the NetBeans, but for the action function, I wrote it by hand. I don't
> understand why you said that I need you all to decompile it ? I don't
> need you to decompile it.

jcranmer@quetzalcoatl /tmp $ unzip NewECGRecord.zip
Archive: NewECGRecord.zip
inflating: NewECGRecord.class
jcranmer@quetzalcoatl /tmp $

It contains only a class file. javap agrees with me that it's a class file.

> First time, I avoid to send the code because I know, it will bothering
> you with that complicated code. So I tried to just describe the flow.
> But you need the original code, so I sent it. But now you said I sent
> too much. Another person said I sent too less. I really don't
> understand what should I provide to make you understand how to solve
> this looping problem.

Provide us with NewECGRecord.java. Not .class, .java.

--
Beware of bugs in the above code; I have only proved it correct, not
tried it. -- Donald E. Knuth

tobleron

unread,
Nov 2, 2008, 9:40:03 AM11/2/08
to

>
> Provide us with NewECGRecord.java. Not .class, .java.
>

I see... here is the .java file http://www.artikelilmiah.com/NewECGRecord-Java.zip
Sorry for the misunderstanding...

RedGrittyBrick

unread,
Nov 2, 2008, 12:04:11 PM11/2/08
to

while ((text = reader.readLine()) != null)
{
contents.append(text)
.append(System.getProperty(
"line.separator"));
jumlahline++;
}

In the above loop, you save the whole signal file in a StringBuffer
which you later convert to a String. You then use Scanner on that String.

I'd not append each line to a StringBuffer. I'd process each line
separately within the above loop. That way you don't have to store the
entire file in some variable, just one record.

I'm assuming it is possible to process one line independently of those
preceding or following it.

P.S. It appears you are using indentation to show the structure of the
output record and not the structure of the program. This makes it hard
to read. I think you could usefully break the program up into smaller
methods and investigate some way of reducing the amount of repetitive code.

--
RGB

Lew

unread,
Nov 2, 2008, 12:04:49 PM11/2/08
to
tobleron wrote:

You forgot to attribute the quote.

Joshua Cranmer wrote:
>> Provide us with NewECGRecord.java. Not .class, .java.

And then tobleron wrote:
> I see... here is the .java file http://www.artikelilmiah.com/NewECGRecord-Java.zip
> Sorry for the misunderstanding...

This is still not an SSCCE, as several of us have advised providing.

> try

For Usenet posts, please use much gentler indentation.

> {
> reader = new BufferedReader(new FileReader(file2));
> String text = null;
>
> // repeat until all lines is read


> while ((text = reader.readLine()) != null)
> {
> contents.append(text).append(System.getProperty( "line.separator"));
> jumlahline++;
> }

...

Here is your problem, just as Martin Gregorie figured. He advised:


> Otherwise, why buffer entire files when you appear to be processing the
> data line by line? Read from the file line by line and output each
> converted line as its completed and your memory problems will vanish.

Of course you're going to run out of memory if you read too much into memory
all at once.

If you insist on reading the entire file into memory, then you will need to
increase -Xmx until it covers your memory needs.

Do provide SSCCEs in the future. You are asking people to help you, it only
makes sense to make it easy for them to do so.

--
Lew

John B. Matthews

unread,
Nov 2, 2008, 1:58:09 PM11/2/08
to
In article <gekmig$a9q$1...@registered.motzarella.org>,
RedGrittyBrick <RedGrit...@spamweary.invalid> wrote:

[...]


> I think you could usefully break the program up into smaller methods
> and investigate some way of reducing the amount of repetitive code.

OP: The urgency of this essential advice will only increase. You might
look to see what else is available:

<http://medical.nema.org/dicom/>

--
John B. Matthews
trashgod at gmail dot com
http://home.roadrunner.com/~jbmatthews/

tobleron

unread,
Nov 3, 2008, 2:54:33 AM11/3/08
to
On Nov 3, 1:04 am, RedGrittyBrick <RedGrittyBr...@spamweary.invalid>
wrote:

> tobleron wrote:
> >> Provide us with NewECGRecord.java. Not .class, .java.
>
> > I see... here is the .java filehttp://www.artikelilmiah.com/NewECGRecord-Java.zip

> > Sorry for the misunderstanding...
>
>    while ((text = reader.readLine()) != null)
>    {
>       contents.append(text)
>       .append(System.getProperty(
>       "line.separator"));
>       jumlahline++;
>    }
>
> In the above loop, you save the whole signal file in a StringBuffer
> which you later convert to a String. You then use Scanner on that String.
>
> I'd not append each line to a StringBuffer. I'd process each line
> separately within the above loop. That way you don't have to store the
> entire file in some variable, just one record.
>
> I'm assuming it is possible to process one line independently of those
> preceding or following it.
>
> P.S. It appears you are using indentation to show the structure of the
> output record and not the structure of the program. This makes it hard
> to read. I think you could usefully break the program up into smaller
> methods and investigate some way of reducing the amount of repetitive code.
>
> --
> RGB

But, I need to write the DICOM VR whith the structure like this : tag;
VR; length; value. The first loop is to find the length (jumlahline++)
and the second loop is to write the value, channel by channel (each
line contains 12 channels). If I use one loop only, how can I write
the length before the value ? The DICOM stream should be written in
sequential. Do you have another suggestion about this ?

Donkey Hottie

unread,
Nov 3, 2008, 5:50:25 AM11/3/08
to
tobleron <bud...@yahoo.com> wrote in
news:b64fa176-8413-4eca...@s1g2000prg.googlegroups.com:

> But, I need to write the DICOM VR whith the structure like this : tag;
> VR; length; value. The first loop is to find the length (jumlahline++)
> and the second loop is to write the value, channel by channel (each
> line contains 12 channels). If I use one loop only, how can I write
> the length before the value ? The DICOM stream should be written in
> sequential. Do you have another suggestion about this ?
>

while ((text = reader.readLine()) != null)
{
  jumlahline++;
}

Add that loop to the beginning, and there you have it.

RedGrittyBrick

unread,
Nov 3, 2008, 6:23:13 AM11/3/08
to

tobleron wrote:
> On Nov 3, 1:04 am, RedGrittyBrick <RedGrittyBr...@spamweary.invalid>
> wrote:
>> tobleron wrote:
>>>> Provide us with NewECGRecord.java. Not .class, .java.
>>> I see... here is the .java filehttp://www.artikelilmiah.com/NewECGRecord-Java.zip
>>> Sorry for the misunderstanding...
>> while ((text = reader.readLine()) != null)
>> {
>> contents.append(text)
>> .append(System.getProperty(
>> "line.separator"));
>> jumlahline++;
>> }
>>
>> In the above loop, you save the whole signal file in a StringBuffer
>> which you later convert to a String. You then use Scanner on that String.
>>
>> I'd not append each line to a StringBuffer. I'd process each line
>> separately within the above loop. That way you don't have to store the
>> entire file in some variable, just one record.
>>
>> I'm assuming it is possible to process one line independently of those
>> preceding or following it.
>>
>> P.S. It appears you are using indentation to show the structure of the
>> output record and not the structure of the program. This makes it hard
>> to read. I think you could usefully break the program up into smaller
>> methods and investigate some way of reducing the amount of repetitive code.

Please delete quoted paragraphs like the above which are not relevant to
your follow-up questions.

>>
>> --
>> RGB

Please don't quote sigs, do delete them when replying.

>
> But, I need to write the DICOM VR whith the structure like this : tag;
> VR; length; value. The first loop is to find the length (jumlahline++)
> and the second loop is to write the value, channel by channel (each
> line contains 12 channels). If I use one loop only, how can I write
> the length before the value ? The DICOM stream should be written in
> sequential. Do you have another suggestion about this ?

In order to understand the above question I'd have to re-read your code.
I am a bit reluctant to do this as it is so large and monolithic.

If you need a "length" of something (line count?) then just read the
file twice, first to count lines (for your preamble for your output
file) and the second time to emit the output records.

This isn't good but it is probably better than storing everything in
memory. Trade I/O for RAM. Measure elapsed time if that becomes an issue.

If the input file has fixed length records (I haven't looked) then you
can just obtain the file size and divide by record length to get the
record count. That way you only need to read the file once.

Another way would be to write the output records to a temp file without
any preamble, accumulate the data for the preamble as you go. At the
end, append the temp file to the preamble. I'm not sure there is any
saving in I/O (it's probably worse) but it may be worth considering.

Sorry if the above suggestions don't match your situation. I'm reluctant
to spend hours of effort on this.

If you can create a concise SSCCE and a good terse description of the
input and output formats then you'll make it easier for people to help you.

John B Matthews' suggestion of prior art seems like an excellent avenue
of research for you.

--
RGB

tobleron

unread,
Nov 3, 2008, 7:48:11 AM11/3/08
to
@All,

I've followed your suggestions, and the problem has been solved. Thank
you very much for your helps.

Best regards.

0 new messages