release candidate of hbamf library with initial WebSockets support
191 views
Skip to first unread message
Aleksander Czajczynski
Jun 11, 2012, 11:39:07 AM6/11/12
to harbou...@googlegroups.com
Hi again!
Here's a candidate for hbamf libary available for download:
http://www.fki.pl/hb/hbamf.tar.gz
http://www.fki.pl/hb/hbamf.zip
It is a few bits of code, so i will wait patiently for all comments and
then sum up some things.
I haven't included any Flash/ActionScript client code, if someone needs
them, just ask me, so i'll add them. This is just, I want to know if
someones interested with the original "point" of this code.
It contains portions of code from Dave Thompson's MIT licensed AmFast C
library for Python. JavaScript AMF classes are taken from node-amf
project - AMF module for NodeJS by Tim Whitlock, available under the MIT
and GPL licenses.
The rest of work was done by me and Ilina Stoilkovska.
Now i would also like to start research for GTNET/GTWEB applications. I
have some proprietary code which acts like GTNET/remote control, so i
have a kind of experience, but it needs to be adapted Harbour Unicode
standards. If someone's skilled in JavaScript is interested in working
out a client/terminal for it, so i'll be more motivated. I would like to
create a GT driver that could inherit from any other GT, this way
GTNUL-inherited one would be for a plain remote access and any other for
remote-control/help/spy capability.
Best regards, Aleksander Czajczynski
Here's a candidate for hbamf libary available for download:
http://www.fki.pl/hb/hbamf.tar.gz
http://www.fki.pl/hb/hbamf.zip
It is a few bits of code, so i will wait patiently for all comments and
then sum up some things.
I haven't included any Flash/ActionScript client code, if someone needs
them, just ask me, so i'll add them. This is just, I want to know if
someones interested with the original "point" of this code.
It contains portions of code from Dave Thompson's MIT licensed AmFast C
library for Python. JavaScript AMF classes are taken from node-amf
project - AMF module for NodeJS by Tim Whitlock, available under the MIT
and GPL licenses.
The rest of work was done by me and Ilina Stoilkovska.
Now i would also like to start research for GTNET/GTWEB applications. I
have some proprietary code which acts like GTNET/remote control, so i
have a kind of experience, but it needs to be adapted Harbour Unicode
standards. If someone's skilled in JavaScript is interested in working
out a client/terminal for it, so i'll be more motivated. I would like to
create a GT driver that could inherit from any other GT, this way
GTNUL-inherited one would be for a plain remote access and any other for
remote-control/help/spy capability.
Best regards, Aleksander Czajczynski
Viktor Szakáts
Jun 11, 2012, 12:04:37 PM6/11/12
to harbou...@googlegroups.com
Hi Aleksander,
> Here's a candidate for hbamf libary available for download:
> http://www.fki.pl/hb/hbamf.tar.gz
> http://www.fki.pl/hb/hbamf.zip
Some comments, focusing on format stuff mostly.
[ Only if the goal is to include this in Harbour SVN. ]
1. I think you should separate WS from HBAMF. HBAMF
should solely contain stuff to serialize/deserialize using
AMF format. This would be pure .c code as I understand.
2. Please add SVN ID and standard copyright headers to all files.
README should be renamed to readme.txt KNOWN.ISSUES
to issues.txt.
3. All hbamf files should use 8.3 short names for
DOS compatibility.
4. I'd recommend running hbformat on all .prg files, otherwise
I will have to do it as the resident cleanup guy. Same for C code,
which needs uncrustify plus config/hb_c.cfg to format it
Harbour stds.
5. For Harbour SVN, bstr.ch is not needed.
6. CHR_AMF3_* macros use plain CHR(), should be hb_BChar(),
it also needs self-inclusion guard.
7. hbref.c uses _HB_API_INTERNAL_, which is wrong for
non-core code.
8. Also the code seems to be missing some
{} enclosure or indentation is wrong.
9. Some or all sources use tabs. All should be replaced
with 3 spaces.
10. Some code uses <stdio.h> and <string.h>, f.e. C RTL
file I/O. These should be replaced with hb_fs*() APIs.
11. C code doesn't cast hb_xgrab() values to proper type,
so it won't build in C++ mode.
12. 'count' should be named s_count in amfio-stdio.c
All static vars should have s_ prefix.
13. I suggest copying .hbc and .hbp files from some
existing Harbour contrib and modify it for hbamf. They
need to be redone.
14. We've been talking about performance, but f.e. some
code call hb_idleSleep( 0.05 ) which will be definitely
slow things down, so I suggest other solution or releasing
idle time only.
15. The lib still depends on hbxpp, maybe you don't use
HB_ISFUNCTION() I just committed.
+1. Next, there is the AMFWS functionality which should be
separate project, f.e. hbwsamf, or simply hbws with AMF
being one build-time or runtime option, with hbamf as
a dependency.
-- Viktor
> Here's a candidate for hbamf libary available for download:
> http://www.fki.pl/hb/hbamf.tar.gz
> http://www.fki.pl/hb/hbamf.zip
[ Only if the goal is to include this in Harbour SVN. ]
1. I think you should separate WS from HBAMF. HBAMF
should solely contain stuff to serialize/deserialize using
AMF format. This would be pure .c code as I understand.
2. Please add SVN ID and standard copyright headers to all files.
README should be renamed to readme.txt KNOWN.ISSUES
to issues.txt.
3. All hbamf files should use 8.3 short names for
DOS compatibility.
4. I'd recommend running hbformat on all .prg files, otherwise
I will have to do it as the resident cleanup guy. Same for C code,
which needs uncrustify plus config/hb_c.cfg to format it
Harbour stds.
5. For Harbour SVN, bstr.ch is not needed.
6. CHR_AMF3_* macros use plain CHR(), should be hb_BChar(),
it also needs self-inclusion guard.
7. hbref.c uses _HB_API_INTERNAL_, which is wrong for
non-core code.
8. Also the code seems to be missing some
{} enclosure or indentation is wrong.
9. Some or all sources use tabs. All should be replaced
with 3 spaces.
10. Some code uses <stdio.h> and <string.h>, f.e. C RTL
file I/O. These should be replaced with hb_fs*() APIs.
11. C code doesn't cast hb_xgrab() values to proper type,
so it won't build in C++ mode.
12. 'count' should be named s_count in amfio-stdio.c
All static vars should have s_ prefix.
13. I suggest copying .hbc and .hbp files from some
existing Harbour contrib and modify it for hbamf. They
need to be redone.
14. We've been talking about performance, but f.e. some
code call hb_idleSleep( 0.05 ) which will be definitely
slow things down, so I suggest other solution or releasing
idle time only.
15. The lib still depends on hbxpp, maybe you don't use
HB_ISFUNCTION() I just committed.
+1. Next, there is the AMFWS functionality which should be
separate project, f.e. hbwsamf, or simply hbws with AMF
being one build-time or runtime option, with hbamf as
a dependency.
-- Viktor
Aleksander Czajczynski
Jun 11, 2012, 1:28:03 PM6/11/12
to harbou...@googlegroups.com
On 2012-06-11 18:04, Viktor Szakáts wrote:
Some comments, focusing on format stuff mostly. [ Only if the goal is to include this in Harbour SVN. ]
[...]
7. hbref.c uses _HB_API_INTERNAL_, which is wrong for non-core code.
[...]
Noted. Thank you.
As for amfio-stdio.c (points 10, 12, 14) it's not used in networking code, so i haven't even touched it, it's not needed in SVN.
Apart from formatting stuff, which i will solve, the important topic to solve is hbref.c and _HB_API_INTERNAL_. I would like to discuss, is it possible to do the same using public API. I'm not interested in pointer to PHB_ITEM, because in Harbour we can have the same value visible under different items. The strings are even more specific case, so the next question if hb_itemGetCPtr( pItem ) for two different items, can eventually point to strings which start the same, but can have different lengths, f.e: "abc", "abcde".
Regards, AC
Noted. Thank you.
As for amfio-stdio.c (points 10, 12, 14) it's not used in networking code, so i haven't even touched it, it's not needed in SVN.
Apart from formatting stuff, which i will solve, the important topic to solve is hbref.c and _HB_API_INTERNAL_. I would like to discuss, is it possible to do the same using public API. I'm not interested in pointer to PHB_ITEM, because in Harbour we can have the same value visible under different items. The strings are even more specific case, so the next question if hb_itemGetCPtr( pItem ) for two different items, can eventually point to strings which start the same, but can have different lengths, f.e: "abc", "abcde".
Regards, AC
Massimo Belgrano
Jun 11, 2012, 1:43:27 PM6/11/12
to harbou...@googlegroups.com
I want notice than Daniel have Uploaded modification to hbsocket with a gtweb proposal
is possible made svn chechout http://code.google.com/p/harbour-websocket/source/checkout to get all source in
(for now is needed copy \harbour\contrib\hbwin\hbwapi.h \harbour\include)
hbmk2 demo.hbp
need be installed wamp with default options, after created a folder ( hbdemp ) in wamp\www ( wamp\www\hbdemo ) and copied inside index.html and harbour generated exe, start wamp, put online & start all service http://127.0.0.1/hbdemo/
2012/6/11 Aleksander Czajczynski <h...@fki.pl>
Hi again!
Here's a candidate for hbamf libary available for download:
http://www.fki.pl/hb/hbamf.tar.gz
http://www.fki.pl/hb/hbamf.zip
Now i would also like to start research for GTNET/GTWEB applications. I have some proprietary code which acts like GTNET/remote control, so i have a kind of experience, but it needs to be adapted Harbour Unicode standards. If someone's skilled in JavaScript is interested in working out a client/terminal for it, so i'll be more motivated. I would like to create a GT driver that could inherit from any other GT, this way GTNUL-inherited one would be for a plain remote access and any other for remote-control/help/spy capability.
Best regards, Aleksander Czajczynski
Massimo Belgrano
Przemysław Czerpak
Jun 11, 2012, 2:21:40 PM6/11/12
to harbou...@googlegroups.com
On Mon, 11 Jun 2012, Aleksander Czajczynski wrote:
Hi,
> Apart from formatting stuff, which i will solve, the important topic
> to solve is hbref.c and _HB_API_INTERNAL_. I would like to discuss,
> is it possible to do the same using public API. I'm not interested
> in pointer to PHB_ITEM, because in Harbour we can have the same
> value visible under different items.
hb_itemSerialize() uses only public API and detects references
to serialized arrays and hashes. As you can see it's not necessary
to use _HB_API_INTERNAL_ to create such code.
> The strings are even more
> specific case, so the next question if hb_itemGetCPtr( pItem ) for
> two different items, can eventually point to strings which start the
> same, but can have different lengths, f.e: "abc", "abcde".
It's impossible because all strings have to end with 0.
In theory it would be possible to share strings like:
"abc"+chr(0)
"abc"+chr(0)+"xyz"
but it's such seldom situation that any code which tries to detect
it would cause global performance reduction.
Now if two pointers returned by hb_itemGetCPtr() are the same then
for sure also string item are equal. Anyhow if you want to reduce
the size of serialized strings then you should make string contents
comparison to store references for repeated strings instead of their
body. But you should also be sure that specification says that they
are deserialized as separated items or as items with copy on write
flag. Otherwise any code which will try to store references to
Harbur string items in serialized data is buggy and this part _HAS_
to be eliminated. In Harbour even if two string items share the
buffer then the buffer has copy on write flag so modifications in
one item do not change other ones sharing the same string buffer.
Such behavior should be restored after deserialization on the second
side.
Anyhow in all cases _HB_API_INTERNAL_ is also completely useless
for string serialization.
best regards,
Przemek
Hi,
> Apart from formatting stuff, which i will solve, the important topic
> to solve is hbref.c and _HB_API_INTERNAL_. I would like to discuss,
> is it possible to do the same using public API. I'm not interested
> in pointer to PHB_ITEM, because in Harbour we can have the same
> value visible under different items.
to serialized arrays and hashes. As you can see it's not necessary
to use _HB_API_INTERNAL_ to create such code.
> The strings are even more
> specific case, so the next question if hb_itemGetCPtr( pItem ) for
> two different items, can eventually point to strings which start the
> same, but can have different lengths, f.e: "abc", "abcde".
In theory it would be possible to share strings like:
"abc"+chr(0)
"abc"+chr(0)+"xyz"
but it's such seldom situation that any code which tries to detect
it would cause global performance reduction.
Now if two pointers returned by hb_itemGetCPtr() are the same then
for sure also string item are equal. Anyhow if you want to reduce
the size of serialized strings then you should make string contents
comparison to store references for repeated strings instead of their
body. But you should also be sure that specification says that they
are deserialized as separated items or as items with copy on write
flag. Otherwise any code which will try to store references to
Harbur string items in serialized data is buggy and this part _HAS_
to be eliminated. In Harbour even if two string items share the
buffer then the buffer has copy on write flag so modifications in
one item do not change other ones sharing the same string buffer.
Such behavior should be restored after deserialization on the second
side.
Anyhow in all cases _HB_API_INTERNAL_ is also completely useless
for string serialization.
best regards,
Przemek
Aleksander Czajczynski
Jun 11, 2012, 3:02:06 PM6/11/12
to harbou...@googlegroups.com
On 2012-06-11 20:21, Przemysław Czerpak wrote:
> On Mon, 11 Jun 2012, Aleksander Czajczynski wrote:
>
> hb_itemSerialize() uses only public API and detects references
> to serialized arrays and hashes. As you can see it's not necessary
> to use _HB_API_INTERNAL_ to create such code.
>
Sure. I have no words to excuse, why i missed hb_hashId() and
> On Mon, 11 Jun 2012, Aleksander Czajczynski wrote:
>
> hb_itemSerialize() uses only public API and detects references
> to serialized arrays and hashes. As you can see it's not necessary
> to use _HB_API_INTERNAL_ to create such code.
>
hb_arrayId(). Many thanks for roadsigns.
> It's impossible because all strings have to end with 0.
> In theory it would be possible to share strings like:
> "abc"+chr(0)
> "abc"+chr(0)+"xyz"
> but it's such seldom situation that any code which tries to detect
> it would cause global performance reduction.
> Now if two pointers returned by hb_itemGetCPtr() are the same then
> for sure also string item are equal. Anyhow if you want to reduce
> the size of serialized strings then you should make string contents
> comparison to store references for repeated strings instead of their
> body.
Thank you for explanation, you just confrimed my little thoughts.
> In theory it would be possible to share strings like:
> "abc"+chr(0)
> "abc"+chr(0)+"xyz"
> but it's such seldom situation that any code which tries to detect
> it would cause global performance reduction.
> Now if two pointers returned by hb_itemGetCPtr() are the same then
> for sure also string item are equal. Anyhow if you want to reduce
> the size of serialized strings then you should make string contents
> comparison to store references for repeated strings instead of their
> body.
Comparsion of string contents is also done during the process. For
example it was necessary to make detection work while serializing data
taken out from RDD.
> But you should also be sure that specification says that they
> are deserialized as separated items or as items with copy on write
> flag. Otherwise any code which will try to store references to
> Harbur string items in serialized data is buggy and this part _HAS_
> to be eliminated. In Harbour even if two string items share the
> buffer then the buffer has copy on write flag so modifications in
> one item do not change other ones sharing the same string buffer.
> Such behavior should be restored after deserialization on the second
> side.
>
I think all is done properly in this matter.
> are deserialized as separated items or as items with copy on write
> flag. Otherwise any code which will try to store references to
> Harbur string items in serialized data is buggy and this part _HAS_
> to be eliminated. In Harbour even if two string items share the
> buffer then the buffer has copy on write flag so modifications in
> one item do not change other ones sharing the same string buffer.
> Such behavior should be restored after deserialization on the second
> side.
>
> Anyhow in all cases _HB_API_INTERNAL_ is also completely useless
> for string serialization.
>
Yes, the function used is public. I just combined two discussions in one.
> for string serialization.
>
Best regards, Aleksander Czajczynski
Aleksander Czajczynski
Jun 13, 2012, 7:28:38 AM6/13/12
to harbou...@googlegroups.com
On 2012-06-11 18:04, Viktor Szakáts wrote:
> 2. Please add SVN ID and standard copyright headers to all files.
> README should be renamed to readme.txt KNOWN.ISSUES
> to issues.txt.
>
Not done. See remarks at bottom of this message.
> README should be renamed to readme.txt KNOWN.ISSUES
> to issues.txt.
>
> 3. All hbamf files should use 8.3 short names for
> DOS compatibility.
>
> 4. I'd recommend running hbformat on all .prg files, otherwise
> I will have to do it as the resident cleanup guy. Same for C code,
> which needs uncrustify plus config/hb_c.cfg to format it
> Harbour stds.
>
Deleted this relationship.
> 6. CHR_AMF3_* macros use plain CHR(), should be hb_BChar(),
> it also needs self-inclusion guard.
>
> 7. hbref.c uses _HB_API_INTERNAL_, which is wrong for
> non-core code.
>
> 8. Also the code seems to be missing some
> {} enclosure or indentation is wrong.
>
> 9. Some or all sources use tabs. All should be replaced
> with 3 spaces.
>
> 10. Some code uses<stdio.h> and<string.h>, f.e. C RTL
> file I/O. These should be replaced with hb_fs*() APIs.
>
> 11. C code doesn't cast hb_xgrab() values to proper type,
> so it won't build in C++ mode.
>
> 12. 'count' should be named s_count in amfio-stdio.c
> All static vars should have s_ prefix.
>
follow these directions. Compiles everywhere, but tested by me only
while paired with Adobe AIR app on Windows.
> 13. I suggest copying .hbc and .hbp files from some
> existing Harbour contrib and modify it for hbamf. They
> need to be redone.
>
> 14. We've been talking about performance, but f.e. some
> code call hb_idleSleep( 0.05 ) which will be definitely
> slow things down, so I suggest other solution or releasing
> idle time only.
>
> 15. The lib still depends on hbxpp, maybe you don't use
> HB_ISFUNCTION() I just committed.
>
> 1. I think you should separate WS from HBAMF. HBAMF
> should solely contain stuff to serialize/deserialize using
> AMF format. This would be pure .c code as I understand.
> +1. Next, there is the AMFWS functionality which should be
> separate project, f.e. hbwsamf, or simply hbws with AMF
> being one build-time or runtime option, with hbamf as
> a dependency.
Separated WS should have choosable serialization method, but i should
> separate project, f.e. hbwsamf, or simply hbws with AMF
> being one build-time or runtime option, with hbamf as
> a dependency.
also work out a basic protocol which wraps it. While maintaning some
more standards now, I vote to not include the lib yet. For separating WS
i need some more spare time. I will keep informing about changes in this
project.
http://www.fki.pl/hb/hbamf.tar.gz
http://www.fki.pl/hb/hbamf.zip
Archives updated.
Overview of some changes not in included in checklist:
* amfdecode.c -> amfdec.c
* amfencode.c -> amfenc.c
* amfio-client.prg -> amfiocli.prg
* amfio-stub.prg -> amfiosrv.prg
* amfio-stdio.c -> amfstdio.c
* amfws-stub.prg -> amfwssrv.prg
* tests/test-client.prg -> examples/client.prg
* tests/test-server.prg -> examples/server.prg
* tests/test-wserver.prg -> examples/wsserver.prg
* js/test-wserver.htm -> js/test-ws.htm
* js/test-wserver.js -> js/test-ws.js
* KNOWN.ISSUES -> issues.txt
* README -> readme.txt
* amfwssrv.prg
* optimized XOR WebSocket unmasking with HB_StrXOR()
* amfiosrv.prg
! fixed recently introduced RTE while handling AMFIO_OBJECTNEW message
+ wsbin.c
* generic .c binary handling routines moved to separate file
* amfio.ch -> hbamf.ch
* js/test-ws.htm
* js/test-ws.js
* added random 48k echo string test
Best regards, Aleksander Czajczyński
Viktor Szakáts
Jun 13, 2012, 8:21:09 AM6/13/12
to harbou...@googlegroups.com
Thanks for your efforts Aleksander. The copyrights and SVN IDs
are still missing, other than that the package is looking very good.
I've updated the .hb* files in attached .zip. Also moved examples
into 'tests' to fit with other contribs. Renamed endian.prg to
tstendin.prg and converted it to use UTF8 source (852 is not
available on many non-Windows tools).
One minor note: pls change all C++ comments to ANSI C ones.
(// -> /* */)
As for WS and AMF functionality separation, I think it's critical,
and should be done before uploading it.
-- Viktor
--
Viktor
are still missing, other than that the package is looking very good.
I've updated the .hb* files in attached .zip. Also moved examples
into 'tests' to fit with other contribs. Renamed endian.prg to
tstendin.prg and converted it to use UTF8 source (852 is not
available on many non-Windows tools).
One minor note: pls change all C++ comments to ANSI C ones.
(// -> /* */)
As for WS and AMF functionality separation, I think it's critical,
and should be done before uploading it.
-- Viktor
Viktor
Viktor Szakáts
Jun 13, 2012, 8:28:01 AM6/13/12
to harbou...@googlegroups.com
Hi Aleksander,
After generating .hbx file, it become visible that some
public function names need to be moved into a namespace:
DYNAMIC BEBIN2U
DYNAMIC BEU2BIN
DYNAMIC BEW2BIN
DYNAMIC LEBIN2U
DYNAMIC LEU2BIN
DYNAMIC LEW2BIN
DYNAMIC OBJAMF
DYNAMIC OBJAMFPROXY
DYNAMIC OBJRPCAMF
DYNAMIC RAWAMF
DYNAMIC XORCHECK
DYNAMIC XORCOUNT
Probably '__AMFWS_' prefix would be the best for these,
though you know better which belongs to where exactly.
BTW, it looks the AMFWS layer still uses mixed LE and
BE values inside one protocol.
-- Viktor
After generating .hbx file, it become visible that some
public function names need to be moved into a namespace:
DYNAMIC BEBIN2U
DYNAMIC BEU2BIN
DYNAMIC BEW2BIN
DYNAMIC LEBIN2U
DYNAMIC LEU2BIN
DYNAMIC LEW2BIN
DYNAMIC OBJAMF
DYNAMIC OBJAMFPROXY
DYNAMIC OBJRPCAMF
DYNAMIC RAWAMF
DYNAMIC XORCHECK
DYNAMIC XORCOUNT
Probably '__AMFWS_' prefix would be the best for these,
though you know better which belongs to where exactly.
BTW, it looks the AMFWS layer still uses mixed LE and
BE values inside one protocol.
-- Viktor
Reply all
Reply to author
Forward
0 new messages