Small program without strings (=GC and multithread)
Dirk Herijgers
We known all that VO has problems with Garbage Collector,
multithreading and 5333 errors.
So, I want to write a program without any GC-object.
My question : Do I use in this small example any GC-Object ?
=======================================
FUNCTION START()
local pMem as PTR
pMem := MemAlloc(10)
MemCopyString(pMem,"1234567890",10)
MemFree(pMem)
RETURN 0
=======================================
Is this a string with is collectable by the GC ? or not ?
Thanks for all reply's
Dirk
Igor Kaliniak
With my humble experience, you can use strings quite safely. Problem will be
arised when you use them (eg) as parameters in vo-functions. Some functions
uses critical section internally, and probably safe; some not.
Unfortunately, we (ok, I) haven't sources of vo-runtime libraries. It is
like black-box: try and see. As to your example, it should be safe, but I
prefer 'local dim' and lstrcpy()<g>.
Igor
Stephen Quinn
Why fight it??
Just don't use ANY dynamic types - Strings, Arrays, Floats, etc...
If you really want to avoid it then use C++ that way it won't matter how you code (as there's no
garbage collector to worry about).
--
HTH
Steve Quinn
FDW
As long as the string is a pseudo-const ( e.g. a define ) or a literal ( as
in your example ) there will never be a problem. These strings ( even when
passed to a function ) are never registered to the GC ( as they are in
'OldSpace' ). When manipulating these string you can use 'Cast2Psz(...)' to
get a pointer to the string ( do not use 'String2Psz()' ). Otherwise when
you want to manipulate the string you should use functions as indicated in
you example and it will be totally safe.
Frans
"Dirk Herijgers" <di...@fujitron.be> wrote in message
news:vn995v48616b1i1ui...@4ax.com...
FDW
Dirk,
In addition to my other post: Use a small structure instead of the string
itself, this will give you additional flexabillity, and you can do
refcounting in the structure so you can know when to free the memory ( and
when not ).
STRUCTURE MyString
MEMBER TheString AS PSZ
MEMBER RefCount AS DWORD
MEMBER SLen AS DWORD
FUNCTION MyStringCreate(pszString AS PSZ) AS MyString PASCAL
LOCAL pMyString AS MyString
pMyString:=MemAlloc(_SizeOf(MyString))
pMyString.RefCount:=1
IF pszString<>NULL_PSZ
pMyString.SLen:=PszLen(pszString )
IF pMyString.SLen<>0
pMyString.TheString:=MemAlloc(pMyString.SLen)
MemCopy(pMyString.TheString,pszString,pMyString.SLen)
ELSE
pMyString.TheString:=NULL_PSZ
END
ELSE
pMyString.SLen:=0
pMyString.TheString:=NULL_PSZ
END
RETURN pMyString
FUNCTION MyStringAddRef(pMyString AS MyString) AS VOID PASCAL
IF pMyString<>NULL_PTR
pMyString.RefCount+=1
END
RETURN
FUNCTION MyStringRelease(pMyString AS MyString) AS VOID PASCAL
IF pMyString<>NULL_PTR
IF pMyString.RefCount<>0
pMyString.RefCount-=1
IF pMyString.RefCount==0
IF pMyString.TheString<>NULL_PSZ
MemFree(pMyString.TheString)
pMyString.TheString:=NULL_PSZ
END
pMyString.SLen:=0
END
END
MemFree(pMyString)
END
RETURN
Frans
"Dirk Herijgers" <di...@fujitron.be> wrote in message
news:vn995v48616b1i1ui...@4ax.com...
Paolo Cherubini
It is possible to extend your nice example to collect many strings:
STRUCTURE MyCollectionOfStrings
MEMBER DIM pString[1] AS MySingleString
STRUCTURE MySingleString
MEMBER TheString AS PSZ
MEMBER RefCount AS DWORD
MEMBER SLen AS DWORD
Ciao
Paolo
"FDW" <fdewit_...@planet.nl> ha scritto nel messaggio
news:b32efs$n3b$1...@reader10.wxs.nl...
Dirk Herijgers
Excuse me, but MS VC++..NET also use GC !!
So, you write : just don't use any dynamic type (like strings)
OK, my question was : use my sample a dynamic type ?
Dirk.
Dirk Herijgers
Dirk
Ginny Caughey
I used code like this in multithreaded apps even before VO 2.5 and if worked
fine. The "1234567890" isn't really a VO string since it's a constant so no
problem.
--
Ginny
"Dirk Herijgers" <di...@fujitron.be> wrote in message
news:vn995v48616b1i1ui...@4ax.com...
Ginny Caughey
> Excuse me, but MS VC++..NET also use GC !!
Yes, but multithreading and dynamic data types work in .Net since .Net was
designed from the beginning to support that, unlike VO which apparently still
uses thread local storage slots when it shouldn't. I don't know if this will
change for 2.7 or not.
Ginny
FDW
Dirk,
As was in my first answer, your example is not using dyn-strings.
Frans
"Dirk Herijgers" <di...@fujitron.be> wrote in message
news:ntp95vgdruri413de...@4ax.com...
FDW
Yes go ahead and 'Make it so'. ;-) I would not use the 'DIM ARRAY', but
maybe a linked list of these structures.
STRUCTURE MyList
MEMBER Previous AS MyList
MEMBER Next AS MyList
MEMBER Data AS MyString
Any way this will complicate things a lot so why bother ;-) keep it simple
and it will work best.
Frans
"Paolo Cherubini" <pa...@ats.it> wrote in message
news:b32npv$fp1$1...@nntp.weblinea.it...
Paolo Cherubini
> Yes go ahead and 'Make it so'.
I did it indeed ;-)
It is very useful and "fast" to sort any kind of data in memory.
Ciao
Paolo
"FDW" <fdewit_...@planet.nl> ha scritto nel messaggio
news:b32v9v$4rf$1...@reader10.wxs.nl...
Geoff Schaller
Go back and use C if you really want to be that primitive!
> MemCopyString(pMem,"1234567890",10)
> Is this a string with is collectable by the GC ? or not ?
Isn't it about time you read the manual? This is a literal string and
therefor it is not collectable. If you are going to delve so deeply into the
nuts and bolts of VO then you areally should read the SDT articles on the GC
and of course the manuals themselves. You shouldn't criticise something you
seem to know very little about.
Geoff
Dirk Herijgers
I'm over-reacting .... on what ? What was my statement ?
But, all together, this was just a small question : Is the sample program using a
string?
And, about reading the manuals ..... I like to read manuals which have correct
information. But, there are lot of things inside the help and inside the manuals
which are not correct, and even more things which are incomplete.
Also, If you critizise everyone who is something asking what can be found in A
manual, then 95% of this NG can not exist anymore.
About GC (and also SDT), I have read these articles many times (I mean really
concentrated reading for 10 or more times), but everyone agree with me that writing a
multithread-app is not possible, without crash. There is nothing about "MT" and "GC".
Also, VO do not spend any article about this.
Maybe you have more information about this.
When is GC coming in live during MT ?
What's the diff between CreateVOThread and CreateThread ?
How does the GC move dynamic objects, which are maybe in use by MT ?
Use the GC any critical section, so we can prevent that GC is starting up.
etc, etc
Dirk
FDW
1st: You can write MT programs, have a look at the 'Black Voodoo' app !
2nd: You need a set of functions like
STRUCTURE TS_PSZ
FUNCTION TS_PSZAddRef(pstruTSPSZ AS TS_PSZ) AS TS_PSZ PASCAL
FUNCTION TS_PSZAssign(pstruDestenation AS TS_PSZ,pszString AS PSZ) AS TS_PSZ
PASCAL
FUNCTION TS_PSZCompare(pstruLeft,pstruRight AS TS_PSZ) AS LONGINT PASCAL
FUNCTION TS_PSZConcat(pstruLeft,pstruRight AS TS_PSZ,lNew:=FALSE AS LOGIC)
AS TS_PSZ PASCAL
FUNCTION TS_PSZCreate(pszString AS PSZ,dwAlloc:=0 AS DWORD) AS TS_PSZ PASCAL
FUNCTION TS_PSZDuplicate(pstruSource AS TS_PSZ) AS TS_PSZ PASCAL
FUNCTION TS_PSZLAt(pstruSource AS TS_PSZ,bMatch AS BYTE) AS DWORD PASCAL
FUNCTION TS_PSZLeft(pstruSource AS TS_PSZ,dwLen AS DWORD,lNew:=FALSE AS
LOGIC) AS TS_PSZ PASCAL
FUNCTION TS_PSZLFind(pstruSource AS TS_PSZ,pstruMatch AS TS_PSZ) AS DWORD
PASCAL
FUNCTION TS_PSZRAt(pstruSource AS TS_PSZ,bMatch AS BYTE) AS DWORD PASCAL
FUNCTION TS_PSZRelease(pstruTSPSZ AS TS_PSZ) AS TS_PSZ PASCAL
FUNCTION TS_PSZRFind(pstruSource AS TS_PSZ,pstruMatch AS TS_PSZ) AS DWORD
PASCAL
FUNCTION TS_PSZRight(pstruSource AS TS_PSZ,dwLen AS DWORD,lNew:=FALSE AS
LOGIC) AS TS_PSZ PASCAL
FUNCTION TS_PSZSubStr2(pstruSource AS TS_PSZ,dwFrom AS DWORD,lNew:=FALSE AS
LOGIC) AS TS_PSZ PASCAL
FUNCTION TS_PSZSubStr3(pstruSource AS TS_PSZ,dwFrom,dwLen AS
DWORD,lNew:=FALSE AS LOGIC) AS TS_PSZ PASCAL
3rd: Make every thing strong typed.
4rd: Use _RegCollNotifyStart() and _RegCollNotifyEnd() to detect GC
(de)activation
Frans
"Dirk Herijgers" <di...@fujitron.be> wrote in message
news:vu7c5vofjc3cfrar2...@4ax.com...
Dirk Herijgers
I will look at Black Voodoo, allthough I do not think to get more information then I
have now.
I am running a MT application, with 30 sockets, 1 listen (accept) socket and 1 gui
thread (= main thread = process).
Each socket received each 5 seconds a message, the message is analysed, then
message is "executed" and the answer is returned.
Its running for 24 hours without any problem, when (!!!) (if)
a thread is answering before another thread get a question. (This situation can be
very easy done, when the client MDI app has only 1 thread, with 30 windows, and each
window has a timer on 5 seconds)
But, the server app crashes after about 1 hour (21600 questions and answer) IF 2 (or
more threads are getting a question on the same time). (You just have to start your
client application on 2 computers, and opens on each computer only 15 windows.)
So, the second example is REALLY MT, because the first example is also MT, but this
is MT what is doing nothing (only executing a select() to test if there is something
to receive). In this second example, only then I have problems with 5333.
BUT, now the question : Your functions looks very well, where to find the sources ?
Thanks Frans,
You really help me
Dirk
FDW
See me at Devfest ;-)
The function headers are taken from one of our commercial applications and I
can not freely distribute then ;-(
In a previous response I have shown how these functions can be developed,
just build on them. Also I will send you more source by private mail.
Frans.
P.s. Have a look at the COMSDK ( part of the VO distro ) it is all static
COM stuff with lots of useful stuff.
"Dirk Herijgers" <di...@fujitron.be> wrote in message
news:q3fc5v886ba9hrfjf...@4ax.com...
Geoff Schaller
> I'm over-reacting .... on what ? What was my statement ?
Yes you are. You were starting a thread to talk about avoiding the GC
without adding the context of why you need to do this. Because without your
issue of multi-threaded apps (which I'm still not convinced about - the app
you describe doesn't appear to be too difficult to manage), there is no need
to avoid the GC. VO is a GCS based environment. What yo uare seeking to do
is quite counter-productive for most applications and quite unecessary. You
will lead newcommers to incorrect conclusions about what is happening here.
And if you have read all the docs you claimed to have read, there is no way
you would have asked if the GC is able to collect a literal string <g>.
I have no trouble with the overall discussion - in fact I find it very
interesting. But context needs to be supplied for the benefit of the many
followers of the thread.
Geoff
- Jari -
> We known all that VO has problems with Garbage Collector,
> multithreading and 5333 errors.
> So, I want to write a program without any GC-object.
Right way to go. <g>
Have you downloaded the code we released for free use.
http://www.codeblock.fi/freetools/coreLite.zip
This should give you some tools to write your stuff in
static memory.
- Jari -
Dirk Herijgers
You are right on most of the points:
I'm not giving all the info about my app. Indeed, at start of this thread, I was not
speaking about MultiT. So, You are right when you see that newcommers will make
incorrect conclusions. And also, I agree with you the VO and the GC is quit stable
when making a non-MT application.
Dirk
Dirk Herijgers
Thanks, I will report when finishing writing the app without dyn objects, if more
then 32 threads are possible
Dirk
On Sun, 23 Feb 2003 22:17:41 +0200, "- Jari -" <Jari....@SPAMREMOVE.codeblock.fi>
wrote:
- Jari -
> Thanks, I will report when finishing writing the app without dyn objects,
if more
> then 32 threads are possible
We have tested this. We believe it doesn't make any difference if threads
are using
only static memory. There still are limits 25b 32 threads and 2.6 64
threads!
- Jari -
Ginny Caughey
unrelated to the GC then (which really is supposed to be thread safe since 2.5.)
--
Ginny
"- Jari -" <Jari....@SPAMREMOVE.codeblock.fi> wrote in message
news:b3ec20$baq$1...@phys-news1.kolumbus.fi...
Geoff Schaller
careful of is context (me too <g>) so that others understand where our
comments might apply and where not.
Cheers.
Geoff
(PS - do you guys go to CttM? ie Belgium guys.)
"Dirk Herijgers" <di...@fujitron.be> wrote in message
news:fsrj5v4s6ugk7o62q...@4ax.com...
Igor Kaliniak
This is definitely GC-limit. Just now I slightly modify Rod's example, and
now I have 96-98 threads. Oh, sorry, philosophers. They eat and eat...<g>
Igor
Ginny Caughey
--
Ginny
"Igor Kaliniak" <rem_i...@gal.ukrpack.net> wrote in message
news:b3g6b1$1ki364$1...@ID-179710.news.dfncis.de...
Igor Kaliniak
it w/o debugger, and 100, 120 threads no problems.
And another (m.b. mad) idea. VO-GC used thread local storage. Number of TLS
indexes was limited to 64 before Win2K. Probably they set this limit in
source code, and any GC-affected func/class will cause GPF for
extra-threads. Ergo: we need sources of runtime libraries in future.
Igor
Caughey" <ginny....@wasteworks.com> wrote in message
news:b3ge1p$1mfnu4$1...@ID-144704.news.dfncis.de...
Ginny Caughey
I agree about the source code, but I don't know if we'll get it. At any rate, it
sounds like increasing the limit shouldn't be hard for Brian's team to do, but
it doesn't seem to me that having a limit at all *should* be necessary, and
changing that could be complicated. So the "bottom line" is that the GC still
isn't thread safe after some magic number of threads, despite what we were told.
--
Ginny
"Igor Kaliniak" <rem_i...@gal.ukrpack.net> wrote in message
news:b3gjo1$1ljmib$1...@ID-179710.news.dfncis.de...
Igor Kaliniak
Sorry, I have some problems with mail server. It's too late by my time, and
I can't call ISP personal. I'll try it later, or you can male me some
ftp-addr. for upload.
Igor
Igor Kaliniak
Dirk Herijgers
Can I have it also ?
Dirk
On Tue, 25 Feb 2003 13:52:35 -0500, "Ginny Caughey" <ginny....@wasteworks.com>
wrote:
Dirk Herijgers
I have already read more then once the term "TLS" or "Thread local storage".
Where I can find info on this item ? Is this Windows or is this VO ?
Dirk
On Tue, 25 Feb 2003 22:28:06 +0200, "Igor Kaliniak" <rem_i...@gal.ukrpack.net>
wrote:
Ginny Caughey
--
Ginny
"Igor Kaliniak" <rem_i...@gal.ukrpack.net> wrote in message
news:b3gpcl$1m0ipm$1...@ID-179710.news.dfncis.de...
> ...solved, ignore it.
>
Ginny Caughey
Thread local storage is Windows. Here's the best reference I found after a quick
search on MSDN:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/thread_local_storage.asp
As you can see, the number of TLS slots is limited depending on the version of
Windows. The question I'm wondering is why the GC needs to use TLS in the first
place.
--
Ginny
"Dirk Herijgers" <di...@fujitron.be> wrote in message
news:3vqn5vgpkphgl1hfa...@4ax.com...
Igor Kaliniak
I still have problems w. mail server, have tried from their web-page. Do you
get it?
Igor
Luigi De Palma
Regards Luigi De Palma
"Igor Kaliniak" <rem_i...@gal.ukrpack.net> ha scritto nel messaggio
news:b3g6b1$1ki364$1...@ID-179710.news.dfncis.de...
Robert van der Hulst
On Tue, 25 Feb 2003, at 18:40:02 [GMT +0200] (which was 17:40 where I live)
you wrote about: 'Small program without strings (=GC and multithread)'
> This is definitely GC-limit. Just now I slightly modify Rod's example, and
> now I have 96-98 threads. Oh, sorry, philosophers. They eat and eat...<g>
If you send me the sample I will talk with Brian and see what I can do
to change/remove the thread limit for VO 2.7.
--
Robert van der Hulst
Dirk Herijgers
Thanks for reply.
I have read it, but do not understand very well
Maybe a few words from you can help me.
Thanks, Dirk
On Tue, 25 Feb 2003 19:04:44 -0500, "Ginny Caughey" <ginny....@wasteworks.com>
wrote:
> Dirk,
Dirk Herijgers
Where can I find it ? some ftp... ?
Dirk
On Wed, 26 Feb 2003 09:19:57 +0200, "Igor Kaliniak" <rem_i...@gal.ukrpack.net>
wrote:
> Dirk,
Igor Kaliniak
In your MB. If not, email me /*remove rem_*/
Igor
- Jari -
> If you send me the sample I will talk with Brian and see what I can do
> to change/remove the thread limit for VO 2.7.
Please also ask him to take a look at the sample I sent him, about
dynamic memory problem with multithreading.
It was sent two weeks ago to bugshunter !
- Jari -
- Jari -
> Windows. The question I'm wondering is why the GC needs to use TLS in the
> first place.
According the Rod, it shouldn't use it at all.
This actually is suggestion Rod made to me in our private emails.
- Jari -
- Jari -
> This is definitely GC-limit. Just now I slightly modify Rod's example, and
> now I have 96-98 threads. Oh, sorry, philosophers. They eat and eat...<g>
Can you please send it to me also!
- Jari -
Dirk Herijgers
Also I do not have yet receive your sample.
Can you please send it to me ?
Thanks a lot.
Dirk
On Thu, 27 Feb 2003 13:19:03 +0200, "- Jari -" <Jari.Sevon@codeblock_SPAMREMOVE_.fi>
wrote:
Dirk Herijgers
Yesterday, I have tried your application with more changes.
One of the biggest changes I have make, is inside the tread.
_DebOut32(PSZ(_CAST,"THREAD STARTED, "+String2Psz("THREAD
ID:"+NTrim(GetCurrentThreadID()))+String2Psz(" TOTAL
THREADS:"+NTrim(GetThreadCount()))))
This is almost your code, but I have added the "TOTAL THREADS"
When I am running now your application, I see that no more then 3 thread are running
on the same time !!!!!
So, I have tried to change your app to let running more threads on the same time, but
it was not be best day, so maby you can change your own program, so that more then 30
threads are running on the same time !!!
After your change, please send me the changed sources.
Dirk, Belgium
Igor Kaliniak
Each new thread registered by GC, it is quite reasonable.
Would be preferable (imho) the next GC-algorithm:
1. M := <number of threads>; n := 0 <g>
2. GC register (unregister) every thread.
3. GC started as a separate thread and suspends threads n+1, ... n +N.
N is OS- dependent, ie
N = min{number of waitable objects - 1, number of thread slots, m.b.
something else}
Of course, each thread stopped in it's safe point, not immediately.
4. /*Extra threads */
If M > (n+N)
n += N
Go To 3. // Or M+=1 and goto 2!
endif
Igor
Dirk Herijgers
I have tested your sources. Crash always on thread 30 !!!
(working with 2.5b-3)
So, you say it could run 95 or 96 threads ?
How ?
Dirk
On Thu, 27 Feb 2003 16:15:06 +0200, "Igor Kaliniak" <rem_i...@gal.ukrpack.net>
wrote:
> Ginny,
- Jari -
> This is definitely GC-limit. Just now I slightly modify Rod's example, and
> now I have 96-98 threads. Oh, sorry, philosophers. They eat and eat...<g>
Sorry, but at least with VO 2.5b-3 your modifeid sample crashes with GPF
after
31 threads. So I really doubt it is capable running more than 64 threads
on 2.6.
Where you are counting simultaneosly executinjg threads in your app???
With or without GC there is thread limit in VO or do you think opposite?
- Jari -
- Jari -
What app you are talking about. I have sent several
samples...?
- Jari -
Igor Kaliniak
Now you prove experimentally that 2.5 limited to 31+1 threads. Also
experimentally proved that 2.6 allows 127+1 threads for GC-free applications
and 64-1-x when you're using _any_ dynamic variable (ex- or implicitly).
>
> Where you are counting simultaneosly executinjg threads in your app???
>
You can see number of threads in caption. If you want check it, use
ProcessViewer.
Igor Kaliniak
Igor
Igor Kaliniak
last point from me (in this thread:).
Igor
Dirk Herijgers
It was your example called "threadCountTest 20020211"
Dirk
On Thu, 27 Feb 2003 22:45:41 +0200, "- Jari -" <Jari....@SPAMREMOVE.codeblock.fi>
wrote:
- Jari -
> Now you prove experimentally that 2.5 limited to 31+1 threads. Also
> experimentally proved that 2.6 allows 127+1 threads for GC-free applications
> and 64-1-x when you're using _any_ dynamic variable (ex- or implicitly).
OK, can't say anything as I haven't installed 2.6 yet. But is sounds strange would have changed
logic, why? IOW I am not saying it is not possible, just that it is strange!
> You can see number of threads in caption. If you want check it, use
> ProcessViewer.
OK, thanks,
- Jari -
FDW
Dirk, and others ;)
This is one line of code found:
>> _DebOut32(PSZ(_CAST,"THREAD STARTED, "+String2Psz("THREAD
>> ID:"+NTrim(GetCurrentThreadID()))+String2Psz(" TOTAL
>> THREADS:"+NTrim(GetThreadCount()))))
This is asking for problems ( how do you come up whit these lines ;-)
_DebOut32(PSZ(_CAST;
,"THREAD STARTED, ";
+String2Psz("THREAD ID:";
+NTrim(GetCurrentThreadID());
);
+String2Psz(" TOTAL THREADS:";
+NTrim(GetThreadCount());
);
);
)
It took me about 5 minutes to figure this out ( and it is total sh*t ). You
can not be serious about code like this.
1: You may never ever cast a dynvar to PSZ !!!
2: You can not add PSZ strings !!!
3: You can not add PSZ strings to STRINGS !!!
4: You should not create dynspace items with no local storage !!!
e.g. do not create dynspace items from within the parameters
These are all dangerous ( especially for GC stability ) and in conflict with
what you are tying to do ( prove the GC instability in relation to treads ).
What you are doing is 'YOU ARE BRAKING THE PROGRAM, AND THAN SAYING, HEY
LOOK THE PROGRAM IS BROKEN'
This is really bad practice and totally unprofessional.
Frans
"Dirk Herijgers" <di...@fujitron.be> wrote in message
news:kjur5v4hhgjflptvc...@4ax.com...
Dirk Herijgers
You are correct.
The sample here is really sh*t, but they say this sample was running without problem.
And indeed, this sample was running.
So, I changes this sample with a little piece of code
"String2Psz(" TOTAL THREADS:"+NTrim(GetThreadCount()))"
in the same bad style the whole program was written.
When I run the program now, I see that maximum 3 threads were running at the same
time, and not the 1000 threads !!!
This was my point.
Dirk
FDW
That may so be, but then something else must be wrong in the example also. I
am not going to dissect the sample, I am just saying that first all code in
the example should be at an acceptable level before any conclusion should be
drawn from it.
Frans
"Dirk Herijgers" <di...@fujitron.be> wrote in message
news:nu076v86l7o8nsdgr...@4ax.com...