Netmail

[Rudi Timmermans]

To: Gooroo
Hi G00r00,

I send again a test netmail to Ward to see if it is fixed now this is his
reply i hope you will understand this i have use google translate ;)

Would be if there is also a MSGID and CHRS kludge-in was good.

Example of a message coming from Bart:

MSGID: 2:292 / 907.1 5239fbd6
CHRS: IBMPC 2

RT> --- Mystic BBS v1.10 A38 (OSX)
RT> * 0rigin: X-TREME BBS - Telnet: xtremebbs.flnet.org: 5432 (2:562 / 140)
RT> string-BY: 562/140 244/1120
RT> P @ TH: 562/140

... tear line, "0rigin", "seen by" and "path" lines do not belong in a
netmail.

---
Vriendelijke Groeten/Greetings,

Rudi Timmermans.

--- Mystic BBS v1.10 A38 (OSX)
* Origin: X-TReMe BBS - Telnet:xtremebbs.flnet.org:5432 (1:1/140)

[g00r00]

To: Rudi Timmermans
RT> ... tear line, "0rigin", "seen by" and "path" lines do not belong in a
RT> netmail.

Thanks. SEEN-BY and PATH was removed from Netmail as mentioned.

I am getting some odd feedback though...

It doesn't make much sense to me to have a MSGID on a Netmail message, but it
*does* make sense to have the option to see the PATH and SEEN-BY.

Likewise, I've now had people tell me MSGID/REPLYID should *NOT* be used in
Netmail, but other people tell me it *has* to have it. I've had the same
happen with origin lines.

I'm going to start requiring a reference to a published FTS documentation at
this point, because a lot of this new feedback seems to be inconsistant which
also makes it incorrect.

--- Mystic BBS v1.10 A38 (Windows)
* Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

[Rudi Timmermans]

To: g00r00
Hi G00r00,

g0> Thanks. SEEN-BY and PATH was removed from Netmail as mentioned.

Ok i have compiled this morning the new source ... Thanks!

g0> I am getting some odd feedback though...

Ok check it out i just tell what Ward told me ...

Thanks for the helps !

[Fireball]

To: g00r00
> From Newsgroup: alt.bbs.mystic

>
> To: Rudi Timmermans
> RT> ... tear line, "0rigin", "seen by" and "path" lines do not belong in a
> RT> netmail.
>
> Thanks. SEEN-BY and PATH was removed from Netmail as mentioned.
>
> I am getting some odd feedback though...
>
> It doesn't make much sense to me to have a MSGID on a Netmail message, but
> it *does* make sense to have the option to see the PATH and SEEN-BY.
>
> Likewise, I've now had people tell me MSGID/REPLYID should *NOT* be used in
> Netmail, but other people tell me it *has* to have it. I've had the same
> happen with origin lines.
>
> I'm going to start requiring a reference to a published FTS documentation
> at this point, because a lot of this new feedback seems to be inconsistant
> which also makes it incorrect.
>

It looks like http://ftsc.org/docs/ has every FTS documentation known to
mankind. lol!

-=Fireball=-
--- Synchronet 3.16a-Linux NewsLink 1.102
* Fireball Express!!! BBS - Amarillo, TX - telnet://fireballex.com

[mark lewis]

To: Rudi Timmermans

On Thu, 19 Sep 2013, Rudi Timmermans wrote to Gooroo:

RT> ... tear line, "0rigin", "seen by" and "path" lines do not belong
RT> in a netmail.

the above is not quite correct... tearlines can be used in all messages...
origin, seen-by and path do not belong in netmail, that is true...

)\/(ark


* Origin: (1:3634/12)

[mark lewis]

To: g00r00

On Thu, 19 Sep 2013, g00r00 wrote to Rudi Timmermans:

g> I am getting some odd feedback though...

g> It doesn't make much sense to me to have a MSGID on a Netmail
g> message,

MSGIDs are used in fidonet for duplicate detection...

g> but it *does* make sense to have the option to see the PATH
g> and SEEN-BY.

in netmail, the VIA line replaces both of those... path and seen-by are only
for echomail...

g> Likewise, I've now had people tell me MSGID/REPLYID should *NOT* be
g> used in Netmail, but other people tell me it *has* to have it. I've
g> had the same happen with origin lines.

i don't know where you have been getting your information but netmail only
differs from echomail by NOT having the AREA, origin, seen-by and path lines...
all other lines are valid in netmail... netmail can also carry VIA lines which
is similar to the seen-by and path lines... VIA is not required but each system
that passes netmail further to its destination should add another VIA to show
that it processed the message and when... VIAs are also used to detect netmail
loops where a netmail message gets routed around in a circle and never gets
directed to the destination for some reason...

g> I'm going to start requiring a reference to a published FTS
g> documentation at this point, because a lot of this new feedback
g> seems to be inconsistant which also makes it incorrect.

FTSC.ORG has all of the specs... unfortunately, not everything widly used has
been raised from proposal to spec... i've tried several times to get this done
on the things we know are widely used but don't get anywhere with it :(

i'm curious as to who told you the guff about the origin, msgid and other stuff
with netmail...

[mark lewis]

To: Rudi Timmermans

On Thu, 19 Sep 2013, Rudi Timmermans wrote to g00r00:

RT> Ok check it out i just tell what Ward told me ...

you should know by now that ward is not always correct ;)

[mark lewis]

To: Fireball

On Thu, 19 Sep 2013, Fireball wrote to g00r00:

> I'm going to start requiring a reference to a published FTS
> documentation at this point, because a lot of this new feedback
> seems to be inconsistant which also makes it incorrect.

F> It looks like http://ftsc.org/docs/ has every FTS documentation
F> known to mankind. lol!

they're supposed to but some are missing... i should have a pretty complete
library of them, though... i've been carrying them since the late 80's or early
90's ;)

FWIW: i'm also a member of the FTSC if that matters to anyone...

[Robert Wolfe]

To: mark lewis
On 09-21-13, mark lewis said the following...

ml> On Thu, 19 Sep 2013, Rudi Timmermans wrote to g00r00:
ml>
ml> RT> Ok check it out i just tell what Ward told me ...
ml>
ml> you should know by now that ward is not always correct ;)

Heh I am not going to even touch this one ;)

--- Mystic BBS v1.10 A38 (Windows)
* Origin: Rivernet BBS | Memphis TN USA | bbs.rivernet.us (1:116/18)

[Ulrich Schroeter]

To: G00r00
Hi G00r00,

Thursday September 19 2013 09:34, you wrote to Rudi Timmermans:

RT>> ... tear line, "0rigin", "seen by" and "path" lines do not belong

RT>> in a netmail.

gr> Thanks. SEEN-BY and PATH was removed from Netmail as mentioned.

gr> I am getting some odd feedback though...

gr> It doesn't make much sense to me to have a MSGID on a Netmail message,

it does, it helps to identify looping netmails for a tracker ...


gr> but it *does* make sense to have the option to see the PATH and
gr> SEEN$BY.

this will be handled differently in FTN routing
each FTN netmail routing system adds a VIA kludge under the netmail

[Ctrl+A]Via 2:244/1120@FidoNet @20130923.013714.11.UTC Itrack+ 2.0 FP000190
ftn aka ^^^^^^^^^^^^^^^^^^ datetime program+version

a path line only has one (or more) lines of aka's listed ..
via lines are each individual single lines, one by one
An echomail that did pass 5 systems has 5 akas listed in one line.
A netmail that did pass 5 systems has 5 VIA kludge lines ...
A netmail is a 1-to-1 so there are no seenby's possible by default
(otherwise its an echomail ...)
If you send a netmail to a CC party, the mail gets copied to each individual
recipient. To 5 recipients the mail has one original and 4 addtl. copies that
gets distributed seperately ... (-> routing)

gr> Likewise, I've now had people tell me MSGID/REPLYID should *NOT* be
gr> used in Netmail,

thats wrong assumption

gr> but other people tell me it *has* to have it. I've
gr> had the same happen with origin lines.

Origin lines are required for echomails including the FTN aka, as the FTN aka
in the origin line will be used in netmail replies to build the destination aka
from

By default there is no Origin lines in netmails, but systems should handle it,
if it is there
to be strict FTN compatible, better to remove the default origin line.

There is an exception, that relates to Areafix and Filemgr requests
here the default is, that the request lines ends with a tearline "---"
without any other following text (!)
sample
From: me 2:244/1120
To : Areamgr 2:244/8001 <-- address Areamgr of 2:244/8001
Subj: password
------------------------------------
%list <-- send list of all available echoes
%query <-- send the list of areas I'm still connected to
%help <-- send help text how to use areamgr
+MYSTIC <-- subscribe to echo MYSTIC
--- <-- tearline, end of request commands


gr> I'm going to start requiring a reference to a published FTS
gr> documentation at this point, because a lot of this new feedback seems
gr> to be inconsistant which also makes it incorrect.

one I've sent you by email last weekend :)

gr> -$- Mystic BBS v1.10 A38 (Windows)
gr> $ Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

regards, uli ;-)

---
* Origin: AMBROSIA - Frankfurt/Main - Germany (2:244/1120)

[Ulrich Schroeter]

To: Mark Lewis
Hi Mark,

Saturday September 21 2013 00:19, you wrote to Rudi Timmermans:

ml> On Thu, 19 Sep 2013, Rudi Timmermans wrote to g00r00:
RT>> Ok check it out i just tell what Ward told me ...

ml> you should know by now that ward is not always correct ;)

LOL =:D

ml> )\/(ark
ml> $ Origin: (1:3634/12)

[g00r00]

To: Ulrich Schroeter
US> it does, it helps to identify looping netmails for a tracker ...

I guess it could help in that area, but I don't think it was meant for that. I
think MSGID/REPLYID was made for reply linking.

Its would be too slow and take too much memory to do a direct comparison of
MSGIDs during dupe checking, so it just becomes another text used in the
calculation of a CRC.

The reason dupe checking is somewhat reliable has little to do with a MSGID;
its due to multiple CRC values being saved for different portions of the
message, then used together for comparison against the incoming message.

If you mean loops as far as a topology loop, then SEEN-BY and Via can do
that. MSGID wouldn't be very useful there either because it only has the
origin address and serial number.

US> this will be handled differently in FTN routing
US> each FTN netmail routing system adds a VIA kludge under the netmail

Thanks, I understand how Via works! :)

As you mentioned in another message I have a placeholder for it in the tosser
I just haven't added it yet because I haven't done the code to get UTC.

US> A netmail is a 1-to-1 so there are no seenby's possible by default
US> (otherwise its an echomail ...)

Mystic doesn't add them, and I understand how it works. I am just suggesting
that it wouldn't have been bad to keep them in by default (back in the
day) because it was already used in echomail.

US> Origin lines are required for echomails including the FTN aka, as the
US> FTN aka in the origin line will be used in netmail replies to build the
US> destination aka from

I have a feeling maybe back 20+ years ago that happened, but now I am not
sure.

Each PKT has a PKT header, and within that each MSG has a MSG header which
contains the original origin of that echomail message. The echomail address
is pulled from that.

If its a v2 PKT it will be missing the zone so you can match it then (possibly
by parsing origin), but even back in the 1990s everyone was using 2.2+ PKT
formats.

I never noticed anything that actually parses the origin line of the message
when a reply-by-netmail is selected! Typically, its pulled from the MSG and
saved into a message base header during import. The software then pulls it
from that when replying by netmail.

--- Mystic BBS v1.10 A38 (Windows)
* Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

[mark lewis]

To: g00r00

On Tue, 24 Sep 2013, g00r00 wrote to Ulrich Schroeter:

US> it does, it helps to identify looping netmails for a tracker ...

g> I guess it could help in that area, but I don't think it was meant
g> for that. I think MSGID/REPLYID was made for reply linking.

not at all... MSGID was created for message identification... REPLY was created
for reply linkage... the two work together but REPLY is only one way (back) to
the original post it is a direct reply to... see FTS-0009 for specifics ;)

g> Its would be too slow and take too much memory to do a direct
g> comparison of MSGIDs during dupe checking, so it just becomes
g> another text used in the calculation of a CRC.

how so too slow and too much memory? this would depend on the structure of the
dupe database and how many values are retained in the dupe database...

g> The reason dupe checking is somewhat reliable has little to do with
g> a MSGID; its due to multiple CRC values being saved for different
g> portions of the message, then used together for comparison against
g> the incoming message.

sounds like a viable plan...

g> If you mean loops as far as a topology loop, then SEEN-BY and Via
g> can do that. MSGID wouldn't be very useful there either because it
g> only has the origin address and serial number.

trackers record these items for later comparison with other messages... since
VIA is not required and not all systems place them in netmail messages, MSGID
offers yet another method to detect routing loops...

US> this will be handled differently in FTN routing each FTN netmail
US> routing system adds a VIA kludge under the netmail

g> Thanks, I understand how Via works! :)

:)

g> As you mentioned in another message I have a placeholder for it in
g> the tosser I just haven't added it yet because I haven't done the
g> code to get UTC.

US> A netmail is a 1-to-1 so there are no seenby's possible by default
US> (otherwise its an echomail ...)

g> Mystic doesn't add them, and I understand how it works. I am just
g> suggesting that it wouldn't have been bad to keep them in by
g> default (back in the day) because it was already used in echomail.

they never existed in netmails and then never existed before echomail was
created...

US> Origin lines are required for echomails including the FTN aka, as

US> the FTN aka in the origin line will be used in netmail replies to
US> build the destination aka from

g> I have a feeling maybe back 20+ years ago that happened, but now I
g> am not sure.

several methods are employed to try to ensure that a netmail message is
addressed to the proper destination... pulling them from the origin line was
the first method before the MSGID was created... when MSGID came along, then it
may have been used in additional to the address in the origin line but that
falls apart when dealing with gated FTN messages... it gets even worse when a
message has been gated multiple times... in gated echomail messages the MSGID
stays the same but the additional origin lines tell the (1) originating system
(2) each additional gate's FTN address... so in those cases, the last valid
origin line and the MSGID will not agree...

then there's the problem of regurgitated messages that have had their MSGID
altered to be that of the regurgitating system... they may or may not carry the
original origin line, too...

g> Each PKT has a PKT header, and within that each MSG has a MSG header
g> which contains the original origin of that echomail message. The
g> echomail address is pulled from that.

what is the destination of an echomail message? if there is only one link, then
the destination would be the link but if there are multiple links, then each
copy of the echomail message will have different destinations... then when the
echomail message gets to the next hop and is packaged for distribution to other
links, the origin is no longer the originating machine but it is now the link
that the message was sent to and the destination is other links from the
link...

here is a packet of two messages in the ALLFIX_FILE echo sent from 1:123/500 to
my main system, 1:3634/12...

=============================================================================
Inf Date From To Subject
=============================================================================
24-Sep-13 Diskshop BBS All New Files At The Diskshop
24-Sep-13 Diskshop BBS All New Files At The Diskshop
=============================================================================
1/2 123/500 3634/12 0000003a ALLFIX_FILE
Type 2+ HPT v1.04 O=1:123/500 D=1:3634/12
=============================================================================


in this last line group, you can see

1. that message 1 of 2 (1/2) is selected
2. the originating address of the selected message is 123/500
3. the destination address of the selected message is 3634/12
4. we are at offset 0000003a of the packet
5. the selected message is in the ALLFIX_FILE echo
6. the packet is a Type 2+ packet
7. the tosser used to create the packet is HPT v1.04
8. the translated (zone added) originating system is 1:123/500
9. the translated (zone added) destination system is 1:3634/12


following are the contents of the selected message above (minus the text body
and with modified control, tear, origin, seenby and path lines to prevent
problems with dumb^H^H^H^Hsome tossers)


=============================================================================
From: Diskshop BBS 123/500 24 Sep 13 12:10:03
To: All 3634/12 ALLFIX_FILE
Subj: New Files At The Diskshop Size: 1879b
Stat: Locl Ofs: 58b
=============================================================================
^ATZUTC: -0400
^AMSGID: 35797.fidonetallfixfi@1:250/501 16a77b30
^APID: SMBUTIL 2.33-Linux r1.103 Oct 30 2012 GCC 4.4.5
^ATID: SBBSecho 2.20-Linux r1.215 Oct 30 2012 GCC 4.4.5
[trim text body]
-!- SBBSecho 2.20-Linux
! Origin: >> diskshop >> bbs.diskshop.ca >> http.telnet.nntp (1:250/501)
$EEN-BY: 12/1 18/0 112/0 116/116 123/0 22 52 57 140 500 789 1025 2000
$EEN-BY: 124/5013 135/364 140/1 154/10 226/600 229/426 250/0 1 8 100
$EEN-BY: 250/308 310 468 501 250/502 261/38 298/5 311/1 320/119 322/759
$EEN-BY: 342/806 387/22 3634/12
^APATH: 250/501 100 123/500
=============================================================================

note the originating system address in the header is not that of the diskshop
bbs, frank's system, as noted in the MSGID and origin lines... it is also the
first entry in the PATH line...


g> If its a v2 PKT it will be missing the zone so you can match it
g> then (possibly by parsing origin), but even back in the 1990s
g> everyone was using 2.2+ PKT formats.

there are three Type 2 formats...

Type 2 (original type 2 format, replaced type 1 format)
Type 2+ (FSC-0039 Sep 1990 Mark Howard)
Type 2.2 (FSC-0045 Apr 1990 Thom Henderson)

i don't know which of 2+ or 2.2 had the highest usage but both were and are
still widely used... sadly, neither have (yet) been raised to a standard by the
FTSC and so they remain proposals in the FTSC Reference library with their
original document numbers of FSC-xxxx...

g> I never noticed anything that actually parses the origin line of the
g> message when a reply-by-netmail is selected! Typically, its pulled
g> from the MSG and saved into a message base header during import.
g> The software then pulls it from that when replying by netmail.

this depends on the software... it is generally not noticible to the user where
the address is drawn from... most users never see the MSGID so the origin line
is the first place they may look to confirm a proper address... and again,
let's also not forget that supporting FTS-0009 is not required so the MSGID may
not exist at all...

[g00r00]

To: mark lewis
ml> not at all... MSGID was created for message identification... REPLY was
ml> created for reply linkage... the two work together but REPLY is only one

Right thats basically what I said. I was giving practical uses for it. As
opposed to it being added for dupe checking netmails - they are used to
identify the message and create reply linkage...

ml> way (back) to the original post it is a direct reply to... see FTS-0009
ml> for specifics ;)

Mystic has had MSGID/REPLYID support for like 10-15 years or so - I don't
really need to. :)

ml> how so too slow and too much memory? this would depend on the structure
ml> of the dupe database and how many values are retained in the dupe
ml> database...

When you are creating a database of huge strings and doing string comparisons,
it is a LOT slower. Like a thousand times slower. As I explained right after
I said that - CRC values are calculated based on text for this reason but
should be doing so at least twice in order to avoid common collision.

The size of the data base doesn't really change how efficient a process is.
Yes, a small database would make it seem okay using string comparison, but
that doesn't make it a good idea or an efficient process. It just makes a bad
process with a small database. :)

ml> they never existed in netmails and then never existed before echomail was
ml> created...

Right thats what I said...

ml> gated echomail messages the MSGID stays the same but the additional
ml> origin lines tell the (1) originating system (2) each additional gate's
ml> FTN address... so in those cases, the last valid origin line and the
ml> MSGID will not agree...

Yep these are all things supporting what I said ... that pulling a reply to
netmail address from the message text is not a good idea these days and
probably would be undesirable. :)

ml> what is the destination of an echomail message? if there is only one
ml> link, then the destination would be the link but if there are multiple

We are talking about origins in the header, not destinations.

The message origin address becomes the destination when you reply by
netmail, and that address is avaiable (in both echo and netmail) in better
places than message text origin lines.

The point is, you wouldn't try to pull the address out of any number of
possibly incorrect origin line texts in the message text - you'd use any of
the more reliable ways to get that value.

ml> i don't know which of 2+ or 2.2 had the highest usage but both were and
ml> are still widely used... sadly, neither have (yet) been raised to a
ml> standard by the FTSC and so they remain proposals in the FTSC Reference

V2 pre-dates zones! I'm shocked that somehow they've never been standardized.

[mark lewis]

To: g00r00

On Tue, 24 Sep 2013, g00r00 wrote to mark lewis:

ml> FTN address... so in those cases, the last valid origin line and

ml> the MSGID will not agree...

g> Yep these are all things supporting what I said ... that pulling a
g> reply to netmail address from the message text is not a good idea
g> these days and probably would be undesirable. :)

i'm not so sure... the last origin line would be used for the destination
address... the gateway to send the response to would be pulled from another
control line in the message... unless the system already had a connection into
that other network... this method is known as zonegating... it is the true and
original zonegating... not what everyone has thought was zonegating ;)

ml> what is the destination of an echomail message? if there is only one
ml> link, then the destination would be the link but if there are multiple

g> We are talking about origins in the header, not destinations.

same difference... the origin in the header of the packet and the echomail
messages packed in the packet is your uplink when viewed from your side of the
fence... the destination is your address in the same view...

g> The message origin address becomes the destination when you reply by
g> netmail,

not to the originating address in the packed echomail message's header...
that's your uplink... the only reliable place to get the originating system's
address from is the origin lines... MSGID is not required... echomail requires
the origin line...

g> and that address is avaiable (in both echo and netmail) in better
g> places than message text origin lines.

/may be available/ in other possibly better place but those other places are
not required... again, the only reliable source is the origin line... that is
what it was created for and how it got its name...

one should note, too, what the tearline is and what it was used for... it was
the point which everything following was torn off and thrown away... so the
mail processor would grab the origin address from the origin line and store it
with other portions of the message (perhaps in a header section) and tear off
the tearline and everything following so that it was not put into the message
base for the users to see or know about as well as making the message area's
data files smaller since the information was not stored...

g> The point is, you wouldn't try to pull the address out of any number
g> of possibly incorrect origin line texts in the message text - you'd
g> use any of the more reliable ways to get that value.

no, you would use the last origin line... if it is not the proper one then a
system upstream that processed the message screwed up and that's a problem to
be taken care of... there's nothing we can do about other problems except point
them out and make them fix them some how... we cannot work around their
problems and we should not do so because that helps to hide the problem which
leads everyone to falsely believe that everything is working AOK... in the
meantime, the developer(s) silently "fixing" others problems are getting more
and more frustrated as they have to keep fitting in unwanted and unnecessary
code to cover other's asses with no compensation... better to let their
mistakes flow to the light were everyone can see them so that they can fix them
themselves and others can remove their unwanted code... talk to joho about this
type of stuff if you get a chance... he spent a lot of time fixing other's
problems and finally stopped that after doing a huge rewrite of frontdoor's
mailer and editor code... then all those problems were flung into the light and
suddenly everyone saw them... they were fixed, finally after all those years,
by the original authors... especially when everyone started raising hell about
them ;)

ml> i don't know which of 2+ or 2.2 had the highest usage but both

ml> were and are still widely used... sadly, neither have (yet) been
ml> raised to a standard by the FTSC and so they remain proposals in
ml> the FTSC Reference

g> V2 pre-dates zones! I'm shocked that somehow they've never been
g> standardized.

yeah, it does... i've also been unable to locate the format of Type 1 packets
and packed messages... i suspect that information may be in the original Fido
BBS stuff but i would probably have better luck locating it if i just ask TJ
about them...

and yes, there's a lot that should have been updated and standardized... seenby
and path lines should be 4D (zones) and maybe 5D (domains)... remember that 3D
is points... i remember the days when NET_DEV (where all this used to be
discussed) was hopping with 200+ messages a day... new proposals being
discussed, implementations, noted problems being worked out, etc... i also
remember a whole lot of political crap being flung about... crap that caused
more arguing than actual discussions and with all that going on, nothing got
done at all... we've had three separate FTSC bodies now and every one of them
has fallen into similar disrepair and nothing gets done :(

[Ulrich Schroeter]

To: G00r00
Hi G00r00,

Tuesday September 24 2013 11:26, you wrote to me:

US>> it does, it helps to identify looping netmails for a tracker ...

gr> I guess it could help in that area, but I don't think it was meant for
gr> that. I think MSGID/REPLYID was made for reply linking.

not only for reply linking ...

Netmail trackers works with MSGID's

from Itrack doc:

command meaning
------------- -------------------------------------------------------
INVKLUDGES If set all Kludges except Msgid, Pid, Vias and
addressing Kludges will be invalidated.

Configuration
MAINADDRESS The Domain is an optional statement for the main domain
name. The domain name is only used in Via and Msgid lines.

While reading messages and no INTL line is in the message, ITRACK
looks for Msgid/Reply kludges to determine the zones. If no zone is
found ITRACK uses the zone of MAINADDRESS.

SETECHOORIGIN Changes the originating address of the mail to the
address found in a Msgid Kludge or " * Origin:" Line.
This feature is usefull if you want to forward
echomail in netmail.

%MSGIDFROM[FIELD]% Address located in the MSGID-kludge

%OLDMSGIDFROM[FIELD]%

With these MSGID variables you can program autoresponders, Votemanagers,
netmail verification and corrections and much more ...


gr> Its would be too slow and take too much memory to do a direct
gr> comparison of MSGIDs during dupe checking, so it just becomes another
gr> text used in the calculation of a CRC.

the msgid consists of a
[[AKA] [+ domain] + [unique id]]
what gets extracted for comparison is the AKA / domain part

the unique id will be used in echomails, as you've said before, for reply
linking
but netmail handling is a bit different ...
similar to the path and via kludge ... via kludge is for debugging purposes,
path probably to detect loops and dupes will be detected by the same kludge
that will be used in netmails too, that is msgid

Netmail | Echomail
Header-Kludges
--------------
INTL Yes | No
MsgID Yes | Yes
Reply Yes | Yes
PID Yes | Yes
CHRS Yes | Yes

Footer-Text
-----------
Origin (Optional)| Yes

Footer-Kludges
--------------
Seenby No | Yes
Path No | Yes
Via Yes | No



gr> The reason dupe checking is somewhat reliable has little to do with a
gr> MSGID; its due to multiple CRC values being saved for different
gr> portions of the message, then used together for comparison against the
gr> incoming message.

by default the message id or crc is to build a unique ID

per FTN spec:
^AMSGID: origaddr serialno

The serialno you can use crc calculation
so for crc calculation you can use
crc( Sender Name + Recipient Name + SenderAKA + RcptAKA + Datetime + random)
or
crc(SenderAKA + Datetime + Random)
or
simply a counter
or whatever calculation schema ...
and has nothing to do with a crc over the message
details in fsc-0041 :)

GENERAL

For best results, MSGID and REPLY lines should be the first two
lines of the message after extended addressing lines (FMPT, TOPT,
INTL, DOMAIN), with MSGID appearing above REPLY.

Using FMPT, TOPT, INTL will only used in netmails ...

gr> If you mean loops as far as a topology loop, then SEEN-BY and Via can
gr> do that. MSGID wouldn't be very useful there either because it only
gr> has the origin address and serial number.

dbridge hasn't created msgid's for a long time and there was a long time
complaining within FTN world to add Msgid to dbridge's mail creation process
...

Netmail headers can be completely rewritten by netmail trackers ..
but in case of an existing Msgid / Reply there exist a strong definition:
a MSGID is generated only at the time of message
creation. An existing MSGID and/or REPLY should never be stripped
from a message passing through an intermediate system. No system
should ever add an MSGID and/or REPLY to, or modify an existing
MSGID / REPLY contained in, a message not originating on that
system.

US>> this will be handled differently in FTN routing
US>> each FTN netmail routing system adds a VIA kludge under the

US>> netmail

gr> Thanks, I understand how Via works! :)

gr> As you mentioned in another message I have a placeholder for it in the
gr> tosser I just haven't added it yet because I haven't done the code to
gr> get UTC.

US>> A netmail is a 1-to-1 so there are no seenby's possible by

US>> default (otherwise its an echomail ...)

gr> Mystic doesn't add them, and I understand how it works. I am just
gr> suggesting that it wouldn't have been bad to keep them in by default
gr> (back in the day) because it was already used in echomail.

the problem with the path and seenby is, its still 2D AKA adressing only.
No Zone info included.
for
140/1 772/1 2432/200
you have to open the nodelist with an editor, to search for host 140 to
identify under which zone this host is located to get a complete 4D AKA
for netmail addressing
1:140/1 3:772/1 2:2432/200
And also datetime when the message has been processed
on a specific node system and by which program is missing in seenby in contrast
to via lines, so the via lines gives much more informations for debug purposes

US>> Origin lines are required for echomails including the FTN aka, as

US>> the FTN aka in the origin line will be used in netmail replies to
US>> build the destination aka from

gr> I have a feeling maybe back 20+ years ago that happened, but now I am
gr> not sure.

gr> Each PKT has a PKT header, and within that each MSG has a MSG header
gr> which contains the original origin of that echomail message.
gr> The echomail address is pulled from that.

this mechanism is for echomails only
doing a netmail reply to an echomail ... aka from origin will be used, correct.


gr> If its a v2 PKT it will be missing the zone so you can match it then
gr> (possibly by parsing origin), but even back in the 1990s everyone was
gr> using 2.2+ PKT formats.

gr> I never noticed anything that actually parses the origin line of the
gr> message when a reply-by-netmail is selected!

default ftn echomail to ftn netmail procedure:

orginal echomail with origin / aka ...
starting a reply-by-netmail .. short -> netmail-reply
the editor uses the logged-in user as sender, uses the main aka that is defined
for the netmail folder, puts the sender from the original echomail as the
netmail recipient and extracts the aka from the origin line of the original
echomail to be used as the new netmails recipients FTN address

the result is a complete netmail message with all components
sender, sender aka
recipient, recipient aka
datetime, flags
subject
text


gr> Typically, its pulled
gr> from the MSG and saved into a message base header during import.

a pkt consists of 2 parts
the packet header ... that includes data about the two parties
the packet is transfered between. The packet header includes Zone definitions.

Part II contains one or more compressed echomails
each with a msg-header, address in 2D (!) AKA adressing format

Since FTN has been enhanced with the 4D addressing, this is not enough to
address FTN recipients. So therefor recipients addressing for echomail replys
has been changed by adding a requirement, to add an Origin line with a 4D AKA
at the end of each individual echomail.

Importing such an echomail from a packet, the part I packet header will be used
in the starting process of the import process: is the packet for us?
is the packet password correct? ok? continue with the import ....

Part II ... echomail ...
ignore or copy msg-header akas ... attribute, cost, datetime, recipient,
sender, subject, text including origin/4D-AKA


gr> The software then pulls it from that when replying by netmail.

using the 2D AKA from the imported echomail you get an incomplete 2D AKA that
you have to complete by analysing the raw nodelist ... from above example, use
node 772/1

search "Host,772," in Nodelist, scroll back in nodelist to previous Zone line,
extract the Zone number
For Host 772 its Zone 3 ...
complete the 2D address with Zone -> 3:772/1
but how can you extract a complete number if the echomail was written by a
point?
No chance by above procedure with nodelist lookup

Well, the msgheader doesn't have a pointnumber field, and echomail has no FMPT
(FroMPoinT) and TOPT (TOPoinT) kludges .....
The complete AKA has to be in the Originline ... remember the
moderators notification about Rudi's incomplete Origin line discussion a few
days ago .-)

Working with Netmails works differently ...
once you have a netmail as source you first check for existance of an INTL and
FMPT and TOPT lines ... this makes the complete Sender / Recipients AKAs
If no INTL, FMPT, TOPT kludge line exists, you can extract the msgheader of
stored netmail message that contains a full 4D AKA (different to stored
echomail with 2D aka!) but old software may fill the netmail addresses header
only with a 2D AKA or may use pointnet(fakenet)/node definition, so you're
better use the INTL / FMPT / TOPT addresses ....


gr> -$- Mystic BBS v1.10 A38 (Windows)
gr> $ Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

[g00r00]

To: Ulrich Schroeter
US> the msgid consists of a
US> [[AKA] [+ domain] + [unique id]]
US> what gets extracted for comparison is the AKA / domain part

Thanks for explaining! I understand how MSGID works. :)

I simply disagree that they are extracted and used for duplicate comparison.
If you know of a tosser that does this please let me know I'd like to see it
and maybe benchmark it!

I can tell you that the tossers I've used do not do that because its too big
and too slow. For sure I can tell you that Crashmail, FastEcho, SBBSecho, do
NOT do this...

They do what Mystic does - they calculate CRC values in multiple places
because the chances of collision with just one calculation is too common.
While MSGID may be included in that calculation of a CRC, its values are not
directly used.

--- Mystic BBS v1.10 A38 (Windows)
* Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

[Ulrich Schroeter]

To: Mark Lewis
Hi Mark,

Wednesday September 25 2013 09:23, you wrote to g00r00:

ml> On Tue, 24 Sep 2013, g00r00 wrote to mark lewis:

ml>> FTN address... so in those cases, the last valid origin line and
ml>> the MSGID will not agree...

g>> Yep these are all things supporting what I said ... that pulling a
g>> reply to netmail address from the message text is not a good idea
g>> these days and probably would be undesirable. :)

ml> i'm not so sure... the last origin line would be used for the
ml> destination address... the gateway to send the response to would be
ml> pulled from another control line in the message... unless the system
ml> already had a connection into that other network... this method is
ml> known as zonegating... it is the true and original zonegating... not
ml> what everyone has thought was zonegating ;)

I've first made a netmail reply to your original mail ... saving the message,
the editor asks me, to send via Zonegate ? => Yes =:)

The result is a netmail with following msgheader:

From: Ulrich Schroeter Orig: 244/1120 26 Sep 13 13:05:19
To: Mark Lewis Dest: 2/1
Re: Netmail 5682b
Attr: Pvt Kill Locl
--------------------------------------------------------------------------

and the msgtext, starting with kludges:

INTL 1:3634/12 2:244/1120
MSGID: 2:244/1120@fidonet 52443116
REPLY: 1:3634/12.0 242dd075
PID: GEDW32 3.0.1
CHRS: IBMPC 2
[full text]
and following last 3 lines of text
.........................................

ml> $ Origin: (1:3634/12)

regards, uli ;-)

.........................................

so no special "ZONEGATE" kludge is placed into the netmail ...
but the msgheader address is set to zonegate aka 2/1

Another problem here: there are no Zonegates listed in Zone 2 Nodelist
for fidonet Zones 1-4, so Zone 1 Zonegate will no longer work ... :-P
but this is another story ....

In the case, a Zonegate were available, the Zonegate picksup the INTL kludge
INTL 1:3634/12 2:244/1120
and forwards the netmail to Zone 1 gateway that forwards the mail to 1:3634/12
probably host routed to 1:3634/0

ml>> what is the destination of an echomail message? if there is only

ml>> one link, then the destination would be the link but if there are
ml>> multiple

g>> We are talking about origins in the header, not destinations.

ml> same difference... the origin in the header of the packet and the
ml> echomail messages packed in the packet is your uplink when viewed from
ml> your side of the fence... the destination is your address in the same
ml> view...

g>> The message origin address becomes the destination when you reply

g>> by netmail,

ml> not to the originating address in the packed echomail message's
ml> header... that's your uplink... the only reliable place to get the
ml> originating system's address from is the origin lines... MSGID is not
ml> required... echomail requires the origin line...

:)

+1

still wrote in one or another mail :)

ml> )\/(ark
ml> $ Origin: (1:3634/12)

[Ulrich Schroeter]

To: G00r00
Hi G00r00,

Tuesday September 24 2013 20:22, you wrote to mark lewis:

ml>> not at all... MSGID was created for message identification...

ml>> REPLY was created for reply linkage... the two work together but
ml>> REPLY is only one
gr> ...

ml>> what is the destination of an echomail message? if there is only

ml>> one link, then the destination would be the link but if there are
ml>> multiple

gr> We are talking about origins in the header, not destinations.

gr> The message origin address becomes the destination when you reply by
gr> netmail, and that address is avaiable (in both echo and netmail) in
gr> better places than message text origin lines.

wrong.

your message you've wrote shows as:

msgheader:
From: g00r00 Orig 261/38 24 Sep 13 20:22:55
To: mark lewis Dest 244/1120
Re: Netmail 3246b
Attr:

So what is 261/38?
Zone 1? Zone 2? Zone 3? Zone 4?
I have to start a raw nodelist lookup to identify net 261 is located in zone 1
and 261/38 isn't g00r00 real destination address !
its my uplinks address!

then have a look in the msgtext that starts with kludges:

TID$ Mystic BBS 1.10 A38
MSGID$ 1:129/215 43389a93 <== probably the Orig aka that I can use
in a netmail reply to reach the
user "g00r00" that is different to
the aka listed in the msgheader
REPLY$ 1:3634/12.0 241b51d9

[the text as we can read it here]

and the message footer:

-$- Mystic BBS v1.10 A38 (Windows)
* $Origin$ Sector 7 [Mystic BBS WHQ] (1:129/215)

> ^^^^^^^^^ Oh Yes, this is the addr
> I can use in NM reply

SEEN$BY$ 1/140 10/1 19/33 34/999 90/1 116/18 120/331 123/500 128/2 187 135/364
SEEN$BY$ 140/1 226/0 160 230/150 244/1120 249/303 250/1 306 261/38 100 1381
SEEN$BY$ 261/1406 266/185 1413 267/152 155 280/1027 292/907 908 311/2 320/119
SEEN$BY$ 340/400 393/68 396/45 633/260 712/848 801/161 189 5030/1256
$PATH$ 129/215 154/10 123/500 261/38


MsgID is optionaly, can be there, but maybe not ...
Origin line "must" be there with a full ftn address


gr> The point is, you wouldn't try to pull the address out of any number
gr> of possibly incorrect origin line texts in the message text - you'd
gr> use any of the more reliable ways to get that value.

msgheader only works if my direct downlink or my direct uplink writes
a mail ... thats also why all the tests Vitus started staying
falsery in my netmail folder :-(
as every message gets it address from Vitus Uplink orig msgheader :-P
This 'must' be changed otherwise Mystic is incapable supporting FTN
netmail correctly

ml>> i don't know which of 2+ or 2.2 had the highest usage but both

ml>> were and are still widely used... sadly, neither have (yet) been
ml>> raised to a standard by the FTSC and so they remain proposals in
ml>> the FTSC Reference

gr> V2 pre-dates zones! I'm shocked that somehow they've never been
gr> standardized.

gr> -$- Mystic BBS v1.10 A38 (Windows)
gr> $ Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

[Rudi Timmermans]

To: Ulrich Schroeter
Hi Uli,

I have here also something wrong with the netmail, when i get a netmail let
say i got a netmail from Ward and i reply on this then it takes the netmail
adress from you 2:244/1120 en also when i look on the header of the message
Ward his netmail adress is the same as above and not his netmail adress...
Has this something to do with the MSGID i dont know .... I hope G00r00 can
fix this soon ....

[mark lewis]

To: Ulrich Schroeter

On Thu, 26 Sep 2013, Ulrich Schroeter wrote to G00r00:

US> The serialno you can use crc calculation
US> so for crc calculation you can use
US> crc( Sender Name + Recipient Name + SenderAKA + RcptAKA +
US> Datetime + random) or
US> crc(SenderAKA + Datetime + Random)
US> or
US> simply a counter
US> or whatever calculation schema ...

Warning, Will Robinson! Warning!

CRCs are easily duped with different data... additionally, you cannot guarantee
no repeats in a three year period...

using a CRC16 or CRC32 for the serialno of a FTN message is a very very very
B.A.D. idea... i haven't heard of CRC64 yet... perhaps it has a larger "key
space" that would result in less possible duplicates but already MD5 is found
to be able to be duplicated with different data...

[g00r00]

To: Rudi Timmermans
RT> I have here also something wrong with the netmail, when i get a netmail
RT> let say i got a netmail from Ward and i reply on this then it takes the
RT> netmail adress from you 2:244/1120 en also when i look on the header of
RT> the message Ward his netmail adress is the same as above and not his
RT> netmail adress... Has this something to do with the MSGID i dont know

I don't know if I am understanding you.

I am able to reply to netmail just fine. The addresses in the header are just
as they should be. Can you give more details so I can try to understand?

Thanks

--- Mystic BBS v1.10 A38 (Windows)
* Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

[Ulrich Schroeter]

To: G00r00
Hi G00r00,

Thursday September 26 2013 07:37, you wrote to me:

US>> the msgid consists of a
US>> [[AKA] [+ domain] + [unique id]]
US>> what gets extracted for comparison is the AKA / domain part

gr> Thanks for explaining! I understand how MSGID works. :)

gr> I simply disagree that they are extracted and used for duplicate
gr> comparison. If you know of a tosser that does this please let me know
gr> I'd like to see it and maybe benchmark it!

again, please split netmail handling and echomail handling.
This are two seperated fields of FTN activity

Tossers handles echomails, Trackers handles netmails

On a simple point installation or endpoint node installation mostly tosser and
tracker are simplyfied combined.

larger systems or host often have the programs separated
Tracker + Tosser

As Tracker I have Itrack running.
Itrack is a modular construction kit, similar a development framework.
you program scripts for checking, selection, bundling, transfering to outbound
one of the trackers task is dupe detection caused by circular paths.
The simple fix, check the via kludges. Most of the netmails playing ping-pong
between two systems can be identified with this method. But this method doesn't
work, if the circular path has a hop in between. So the next possible choice to
detect a netmail routing loop is detection by a unique identifier, that is the
msgid ...

For a mystic system this has no relevance as mystic probably will never act as
a netmail host. But there are other systems around that takes care off. So if
you want to make the Mystic package FTN friendly and compatible, so you add
such features, that makes life easier for others ....


gr> I can tell you that the tossers I've used do not do that because its
gr> too big and too slow. For sure I can tell you that Crashmail,
gr> FastEcho, SBBSecho, do NOT do this...

I don't talk about tossers ... they'll dumb support netmails at the lowest
level

in real life you have a similar construct in emailing. Compare a
big companies internet email gateway / mailexchanger and a small companies
own user email server .. there is a big big difference of both installations


gr> They do what Mystic does - they calculate CRC values in multiple
gr> places because the chances of collision with just one calculation is
gr> too common. While MSGID may be included in that calculation of a CRC,
gr> its values are not directly used.

common practice is that mostly all netmail creation software adds a msgid while
mail exchanger systems with trackers uses it in processing (to determine the
full origin senders aka instead of an origin line in echomails).

you are correct saying, that not all systems supports such a processing ...

[Ulrich Schroeter]

To: Rudi Timmermans
Hi Rudi,

Thursday September 26 2013 19:38, you wrote to me:

RT> Hi Uli,
RT> I have here also something wrong with the netmail, when i get a
RT> netmail let say i got a netmail from Ward and i reply on this then it
RT> takes the netmail adress from you 2:244/1120

If I get it correct, the reason is, 'cause mystic extracts ftn addresses
from the msgheader ... but this is a false assumption as the msgheader has the
last transfer addresses included ...

sample: originator of the netmail was 2:292/854. The netmail was routed via
2:244/1120 and delivered to 2:562/140, so the msgheader's orig/dest aka is set
to orig: 244/1120 and dest: 562/140 (note the 2D addressing here !!!)

Mystic extracts on reply the msgheader aka and gets as destination system aka
244/1120
a. the zone is missing
b. the destination is wrong, as it is the last system, that has delivered
the mail and not the origin nodes aka !

On netmail reply to a netmail, there are 2 options:
use the INTL,TOPT,FMPT kludge or MSGID kludge
extracting the recipients aka to address the correct destination


RT> en also when i look on the header of the message Ward his netmail
RT> adress is the same as above and not his netmail adress...

thats because the tossers and trackers puts the direct links aka's
into the binary msgheaders.

your original netmail sent includes in the msgheader
orig: 562/140 dest: 244/1120 before it gets delivered to 2:244/1120
once it is delivered, my tracker changes the routing based on routing
informations and for destination 2:292/854 extracted from the INTL,FMPT,TOPT or
MSGID info of the netmail in processing to Orig: 244/1120 Dest: 292/854
the INTL,FMPT,TOPT,MSGID untouched.


RT> Has this
RT> something to do with the MSGID

it may relate to this problem in the case no correct INTL,FMPT,TOPT kludges are
included in those netmails ....


RT> i dont know .... I hope G00r00 can fix this soon ....


RT> -$-
RT> Vriendelijke Groeten/Greetings,
RT> Rudi Timmermans.
RT> -$- Mystic BBS v1.10 A38 (OSX)
RT> $ Origin: X-TReMe BBS - Telnet:xtremebbs.flnet.org:5432 (1:1/140)

[Ulrich Schroeter]

To: Mark Lewis
Hi Mark,

Thursday September 26 2013 19:15, you wrote to me:

ml> On Thu, 26 Sep 2013, Ulrich Schroeter wrote to G00r00:

US>> The serialno you can use crc calculation
US>> so for crc calculation you can use
US>> crc( Sender Name + Recipient Name + SenderAKA + RcptAKA +
US>> Datetime + random) or
US>> crc(SenderAKA + Datetime + Random)
US>> or
US>> simply a counter
US>> or whatever calculation schema ...

ml> Warning, Will Robinson! Warning!

ml> CRCs are easily duped with different data... additionally, you cannot
ml> guarantee no repeats in a three year period...

ml> using a CRC16 or CRC32 for the serialno of a FTN message is a very
ml> very very B.A.D. idea... i haven't heard of CRC64 yet... perhaps it
ml> has a larger "key space" that would result in less possible duplicates
ml> but already MD5 is found to be able to be duplicated with different
ml> data...

fts-0009 only describes the syntax
^MSGID: origaddr serialno
but not how the serialno has to be constructed

There is a good description in fidonews FIDO711.NWS (12 Mar 1990)
from jim nutt

^MSGID: zone:net/node.point@domain serialno

It is not the intent to limit the address field of the msgid to
fidonet style addresses, they are used here simply for
illustration, messages originating from other types of systems
should use an address appropriate to the originating system. The
serial number may be anything the developer's little heart
desires, AS LONG AS IT IS UNIQUE, NO TWO MESSAGES ON A SYSTEM MAY
SHARE A SERIAL NUMBER!! The serial number is formatted as an 8
character hexadecimal integer, i.e. a 32 bit word. since this
yields in the neighborhood of 4 billion unique numbers, I don't
think it will be a limit for most systems. A common choice for
the serial number is the number of seconds since 1 jan 1970
00:00:00 gmt, this is unique on a single user system and
relatively easy to generate. A more elaborate system will
doubtless be necessary in the case of a multi-user system.

ml> )\/(ark

ml> $ Origin: (1:3634/12)

[Ulrich Schroeter]

To: G00r00
Hi G00r00,

Friday September 27 2013 10:07, you wrote to Rudi Timmermans:

RT>> I have here also something wrong with the netmail, when i get a

RT>> netmail let say i got a netmail from Ward and i reply on this
RT>> then it takes the netmail adress from you 2:244/1120 en also when
RT>> i look on the header of the message Ward his netmail adress is
RT>> the same as above and not his netmail adress... Has this
RT>> something to do with the MSGID i dont know

gr> I don't know if I am understanding you.

gr> I am able to reply to netmail just fine. The addresses in the header
gr> are just as they should be. Can you give more details so I can try to
gr> understand?

first of all, currently we're talking about NETMAIL

below is a sample message that Vitus wrote:

From: Vitus Zeel 244/8001 26 Sep 13 09:15:13
To: Ulrich Schroeter 244/1120
Re: Offline 881b
Attr: Pvt Rcvd

The body text includes following kludge lines:
INTL 2:244/1120 2:244/8001
TID: Mystic BBS 1.10 A38

These are the two places where FTN addresses exist:

header Orig AKA -> 244/8001 (2D format !)

> ^^^^^^^^

Dest AKA -> 244/1120 (2D format !)

kludge INTL 2:244/1120 2:244/8001

> ^^^^^^^^^^

On a netmail reply to this netmail, where do you pickup the FTN address to
address the destination system ?
a. from the message header ? or
b. from the INTL kludge line ?

(origin line doesn't count in netmails, as no FTN system supports Origin lines
in netmails !)


only answer b. will be correct!

A similar message that Rudi has received from Ward looks as:

From: Ward Dossche 244/1120 26 Sep 13 09:15:13
To: Rudi Timmermans 562/140
Re: some text 881b
Attr: Pvt Rcvd

The body text includes following kludge lines:
INTL 2:562/140 2:292/854
MSGID 2:292/854@fidonet 12345678
TID: ...

These are the three places where FTN addresses exist:

header Orig AKA -> 244/1120 (2D format !)

> ^^^^^^^^

Dest AKA -> 562/140 (2D format !)

kludge INTL 2:562/140 2:292/854

> ^^^^^^^^^

kludge MSGID 2:292/854@fidonet 12345678

> ^^^^^^^^^

On a netmail reply to this netmail, where do you pickup the FTN address to
address the destination system ?
a. from the message header ? or
b. from the INTL kludge line ? or
c. from the MSGID kludge line ?


only answer b. + c. will be correct!


Why ?

well ... here is the link structure:

2:292/854 writes a netmail to 2:562/140 but routes it via 2:244/1120

so address gets burned into the msgheader as orig aka on the last way to
2:562/140
so this shows the message header:
Sendername Ward Orig AKA 244/1120
but the INTL kludge and MSGID kludge tells otherwise
the orig system is 2:292/854

In a netmail reply using the msgheader aka, the sendername becomes the
recipients name -> Ward with destination aka 244/1120
?!?!?!? => FAIL!

if the destination AKA will be picked up from the INTL kludge or MSGID kludge
instead, it will correctly address the recipients final destination address :)

Other question: why I had tons of netmails from Rudi and Vitus trying to
netmail reply foreign AKA users, but their netmails ended up in my netmail
folder that is responsible for the AKA 2:244/1120 ?!?!?
In all these cases I've returned the netmails to the senders, that the netmails
gots stuck in my netmail folder, as the intendend users aren't active on my
system .....

don't rely on the msgheader Orig and Dest AKAs (this is what all
FTN documentations says)

use INTL || MSGID kludge if you handle a netmail source and
use Origin line AKA if you have to reply to an echomail
for getting the correct recipients aka


gr> Thanks

gr> -$- Mystic BBS v1.10 A38 (Windows)
gr> $ Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

[Rudi Timmermans]

To: g00r00
Hi g00r00!

27 Sep 13 10:07, you wrote to me:

gr> I don't know if I am understanding you.

gr> I am able to reply to netmail just fine. The addresses in the header
gr> are just as they should be. Can you give more details so I can try to
gr> understand?

G00r00 when i got here a netmail from let say 2:292/854 and then i reply on in
in Mystic then it take me to my hub adress and not 2:292/854 so i need to day
every time a nodelist search for the correct adress to send to. I hope you
understand it now ?

On my system the adress is not correct it is the adress of my Hub not from the
node that has send me the message, i dont know if this will have to do
something with OSX ...

-+-
Vriendelijke Groeten/Greetings,

Rudi Timmermans.

--- ---
* Origin: OS X/Golded-NSF/CrashMail (1:1/140)

[mark lewis]

To: g00r00

On Thu, 26 Sep 2013, g00r00 wrote to Ulrich Schroeter:

g> Thanks for explaining! I understand how MSGID works. :)

g> I simply disagree that they are extracted and used for duplicate
g> comparison. If you know of a tosser that does this please let me
g> know I'd like to see it and maybe benchmark it!

fastecho uses the MSGID for dupe detection... it also uses a CRC routine but i
do not know what sections of the header and message are used in the
calculation...

g> I can tell you that the tossers I've used do not do that because its
g> too big and too slow. For sure I can tell you that Crashmail,
g> FastEcho, SBBSecho, do NOT do this...

yes, fastecho does... hang on a second and let me see if i can locate a
duplicate message in my dupe base...

[time passes]

dang it... i've only got one dupe right now and it was detected by CRC32 but i
do see some that are caught by MSGID... how can i tell this? because fastecho
puts a CHECKED control line in the message along with the system address that
send the apparent dupe...

here's the "notes" (as frontdoor's editor calls them) from the only dupe
message i currently have...

AREA:ALLFIX_FILE
FROM: 1:123/500
CHECKED: CRC32
PID: ALLFIX+ 6.00.18 EFF59863
TID: FastEcho 1.46.1 43246
SEEN-BY: 18/0 112/0 116/116 123/0 22 52 57 140 500 789 1025 2000
SEEN-BY: 124/5013 135/364 140/1 154/10 226/600 229/426 250/100
SEEN-BY: 261/38 298/5 311/1 320/119 322/759 342/806 387/22 3634/12
PATH: 123/500

those detected via MSGID will have MSGID in the CHECKED: control line... i will
keep an eye out and post one for you the next time i see one and remember this
thread ;)

g> They do what Mystic does - they calculate CRC values in multiple
g> places because the chances of collision with just one calculation is
g> too common. While MSGID may be included in that calculation of a
g> CRC, its values are not directly used.

sorry...

[mark lewis]

To: Ulrich Schroeter

On Thu, 26 Sep 2013, Ulrich Schroeter wrote to Mark Lewis:

US> I've first made a netmail reply to your original mail ... saving
US> the message, the editor asks me, to send via Zonegate ? => Yes =:)

US> The result is a netmail with following msgheader:

US> From: Ulrich Schroeter Orig: 244/1120 26 Sep 13 13:05:19
US> To: Mark Lewis Dest: 2/1
US> Re: Netmail
US> 5682b Attr: Pvt Kill Locl
US> -------------------------------------------------------------------

yes, because you said to zonegate it (there are no more zonegates, though and
their concept was not properly understood by many)...

US> and the msgtext, starting with kludges:

US> INTL 1:3634/12 2:244/1120
US> MSGID: 2:244/1120@fidonet 52443116
US> REPLY: 1:3634/12.0 242dd075
US> PID: GEDW32 3.0.1
US> CHRS: IBMPC 2

yes... that is correct...

US> [full text]
US> and following last 3 lines of text
US> .........................................

ml> $ Origin: (1:3634/12)

US> regards, uli ;-)
US> .........................................

US> so no special "ZONEGATE" kludge is placed into the netmail ... but
US> the msgheader address is set to zonegate aka 2/1

i never implied that there was any kind of zongate related control line...

US> Another problem here: there are no Zonegates listed in Zone 2
US> Nodelist for fidonet Zones 1-4, so Zone 1 Zonegate will no longer
US> work ... :-P but this is another story ....

yes...

US> In the case, a Zonegate were available, the Zonegate picksup the
US> INTL kludge INTL 1:3634/12 2:244/1120
US> and forwards the netmail to Zone 1 gateway that forwards the mail
US> to 1:3634/12

yes, this is how it used to work... this is, actually, how it continues to work
but dedicated zonegates are not used today... this does cause a problem geting
between FTN networks, though :/

US> probably host routed to 1:3634/0

routing depends on the gateway... this day in time, probably not host routed...
more likely echomail routed... it is faster and easier since the netmail simply
travels with your echomail when you make the connection for it...

here in Z1, i'd have to make an eWAG that there is maybe possibly 1% of netmail
that is host routed... that guesstimate is extremely generous, too ;)

[mark lewis]

To: Ulrich Schroeter

On Fri, 27 Sep 2013, Ulrich Schroeter wrote to G00r00:

US> again, please split netmail handling and echomail handling.
US> This are two seperated fields of FTN activity

US> Tossers handles echomails, Trackers handles netmails

not over here they do not... trackers track and nothing else... tossers toss
the mail into and scan it out of the bbs' message bases...

my ot-track only tracks and msgtrack before it only tracks... but then, i
really don't need either since my mailer and tosser add the VIA lines for
tracking all by themselves...

i think maybe the only useful function i get from a netmail tracker is bouncing
of unwanted netmail but it has been so long since i've needed that that i don't
recall...

[mark lewis]

To: Ulrich Schroeter

On Fri, 27 Sep 2013, Ulrich Schroeter wrote to G00r00:

US> As Tracker I have Itrack running.
US> Itrack is a modular construction kit, similar a development
US> framework. you program scripts for checking, selection, bundling,
US> transfering to outbound one of the trackers task is dupe detection
US> caused by circular paths. The simple fix, check the via kludges.
US> Most of the netmails playing ping-pong between two systems can be
US> identified with this method. But this method doesn't work, if the
US> circular path has a hop in between.

hunh? you cannot detect a circular VIA path that has more than two systems in
it? you should be able to easily detect a circular path with a hundred systems
in it... it isn't that hard to do...

[mark lewis]

To: Ulrich Schroeter

ml> Warning, Will Robinson! Warning!

ml> CRCs are easily duped with different data... additionally, you

ml> cannot guarantee no repeats in a three year period...

ml> using a CRC16 or CRC32 for the serialno of a FTN message is a very
ml> very very B.A.D. idea... i haven't heard of CRC64 yet... perhaps

ml> it has a larger "key space" that would result in less possible
ml> duplicates but already MD5 is found to be able to be duplicated
ml> with different data...

US> fts-0009 only describes the syntax
US> ^MSGID: origaddr serialno
US> but not how the serialno has to be constructed

true...

US> There is a good description in fidonews FIDO711.NWS (12 Mar 1990)
US> from jim nutt

yeah... kinda and not...

US> ^MSGID: zone:net/node.point@domain serialno

US> It is not the intent to limit the address field of the msgid
US> to fidonet style addresses, they are used here
US> simply for illustration, messages originating from other
US> types of systems should use an address appropriate to the
US> originating system. The serial number may be anything the
US> developer's little heart desires, AS LONG AS IT IS UNIQUE,
US> NO TWO MESSAGES ON A SYSTEM MAY SHARE A SERIAL NUMBER!!

not /on/ a system but _from_ a system... the entire control line contents are
used for the detection... address and serial number...

US> The serial number is formatted as an 8 character hexadecimal
US> integer, i.e. a 32 bit word. since this yields in the
US> neighborhood of 4 billion unique numbers, I don't think it
US> will be a limit for most systems. A common choice for the
US> serial number is the number of seconds since 1 jan 1970
US> 00:00:00 gmt, this is unique on a single user system and
US> relatively easy to generate.

this limits a system to only one message per second or maybe one message per
two seconds... much too slow...

US> A more elaborate system will doubtless be necessary in the
US> case of a multi-user system.

there is that, too...

however, i have posting software that easily generates several hundred messages
per second... with no duplicated serial numbers... so, yeah, as noted above,
kinda and not...

there is also a recent proposal in the FTSC library concerning the serial
number generation... it is close to what i use but what i use was written back
in the '90s and is also multinode (bbs node) usable...

but then again, one doesn't really need anything fancy to use a simply
incrementing counter that starts at zero, do they? ;)

[Ulrich Schroeter]

To: Mark Lewis
Hi Mark,

Saturday September 28 2013 15:45, you wrote to me:

ml> On Thu, 26 Sep 2013, Ulrich Schroeter wrote to Mark Lewis:

US>> I've first made a netmail reply to your original mail ... saving
US>> the message, the editor asks me, to send via Zonegate ? => Yes

US>> =:)

US>> The result is a netmail with following msgheader:

US>> From: Ulrich Schroeter Orig: 244/1120 26 Sep 13 13:05:19

US>> To: Mark Lewis Dest: 2/1 Re: Netmail

US>> 5682b Attr: Pvt Kill

US>> Locl ------------------------------------------------------------
US>> -------

ml> yes, because you said to zonegate it (there are no more zonegates,
ml> though and their concept was not properly understood by many)...

US>> and the msgtext, starting with kludges:

US>> INTL 1:3634/12 2:244/1120
US>> MSGID: 2:244/1120@fidonet 52443116
US>> REPLY: 1:3634/12.0 242dd075
US>> PID: GEDW32 3.0.1
US>> CHRS: IBMPC 2

ml> yes... that is correct...

US>> [full text]
US>> and following last 3 lines of text
US>> .........................................

ml>> $ Origin: (1:3634/12)

US>> regards, uli ;-)
US>> .........................................

US>> so no special "ZONEGATE" kludge is placed into the netmail ...

US>> but the msgheader address is set to zonegate aka 2/1

ml> i never implied that there was any kind of zongate related control
ml> line...


from the original mail I've responded to:

....snip_on......................................................
- INT-M-MYSTIC (2:244/1120) ------------------------------------------ MYSTIC -
Msg : #310 [378] -308 +314 Uns Loc
From : Ulrich Schroeter 2:244/1120 Thu 26 Sep 13 13:09
To : Mark Lewis Thu 26 Sep 13 16:58
Subj : Netmail
-------------------------------------------------------------------------------
Hi Mark,

Wednesday September 25 2013 09:23, you wrote to g00r00: <==== original

ml> On Tue, 24 Sep 2013, g00r00 wrote to mark lewis:

ml>> FTN address... so in those cases, the last valid origin line and
ml>> the MSGID will not agree...

g>> Yep these are all things supporting what I said ... that pulling a
g>> reply to netmail address from the message text is not a good idea
g>> these days and probably would be undesirable. :)

ml> i'm not so sure... the last origin line would be used for the
ml> destination address... the gateway to send the response to would be
ml> pulled from another control line in the message... unless the system
ml> already had a connection into that other network... this method is
ml> known as zonegating... it is the true and original zonegating... not
ml> what everyone has thought was zonegating ;)

I've first made a netmail reply to your original mail ... saving the message,

the editor asks me, to send via Zonegate ? => Yes =:)

[....]

ml> )\/(ark

ml> $ Origin: (1:3634/12)

regards, uli ;-)

-$-
$ Origin: AMBROSIA - Frankfurt/Main - Germany (2:244/1120)

....snip_off.....................................................

ml> destination address... the gateway to send the response to would be
ml> pulled from another control line in the message... unless the system

=> pulled from another control line


my reply results in message that lists the

US>> REPLY: 1:3634/12.0 242dd075

that derived from the previous MSGID line thats

=> pulled from another control line

and addtl. to explain g00r00 that there is no zonegate kludge line

The zone information (-> source of original mail was 1:3634/12.0
from original msgid kludge or in case of a netmail extracted from the intl
kludge or in case of an echomail extracted from the origin line
is Zone 1 ... translated into the Zonegate info
own zone -> 2 slash -> destination zone extracted from msgid -> 1
makes Zonegate 2/1

US>> Another problem here: there are no Zonegates listed in Zone 2
US>> Nodelist for fidonet Zones 1-4, so Zone 1 Zonegate will no longer
US>> work ... :-P but this is another story ....

ml> yes...

US>> In the case, a Zonegate were available, the Zonegate picksup the
US>> INTL kludge INTL 1:3634/12 2:244/1120
US>> and forwards the netmail to Zone 1 gateway that forwards the mail
US>> to 1:3634/12

ml> yes, this is how it used to work... this is, actually, how it
ml> continues to work but dedicated zonegates are not used today... this
ml> does cause a problem geting between FTN networks, though :/

in zone2 nodelist there are still 2 (or 3?) foreign fidonet zonegates listed
zone 9 (virnet) and zone 16 (zyxelnet). there is another listing #33 (MasCanc),
but I cannot identify this as a zonegate for zone 33 ...

US>> probably host routed to 1:3634/0

ml> routing depends on the gateway... this day in time, probably not host
ml> routed... more likely echomail routed... it is faster and easier since
ml> the netmail simply travels with your echomail when you make the
ml> connection for it...

ml> here in Z1, i'd have to make an eWAG that there is maybe possibly 1%
ml> of netmail that is host routed... that guesstimate is extremely
ml> generous, too ;)

ml> )\/(ark

ml> $ Origin: (1:3634/12)

[Ulrich Schroeter]

To: Mark Lewis
Hi Mark,

Saturday September 28 2013 16:18, you wrote to me:

ml> On Fri, 27 Sep 2013, Ulrich Schroeter wrote to G00r00:

US>> again, please split netmail handling and echomail handling.
US>> This are two seperated fields of FTN activity

US>> Tossers handles echomails, Trackers handles netmails

ml> not over here they do not... trackers track and nothing else...
ml> tossers toss the mail into and scan it out of the bbs' message
ml> bases...

tossers are dumb silly echomail shufflers ... they are optimized for.
most tossers also supports netmail packaging, but this isn't the tossers
main area and on complex netmail handling they'll fail
so tossers are mostly included in point packets

mail exchangers (hosts, and probably also echomail hubs) using trackers
for netmail processing 'cause they can support node and pointlists (that
tossers seldom support)

crashmail, routed mail, nodelist lookups, netmail bouncing, host- and
hub-routing, subgroup routing ...
dynamic outbound queue mailers like frontdoor routes mails according to routing
configs and events. binkley + binkd with a static outbound requires another
tool that handles netmail routing. This task is accomplished by Trackers.

HPT (husky) is one of the last bigger developments in this area is a hybrid
system, with tosser, ticker and tracker. HPT supports routing definition like a
tracker. Crashmail is another development in this direction that supports
routing definition.

ml> my ot-track only tracks and msgtrack before it only tracks... but
ml> then, i really don't need either since my mailer and tosser add the
ml> VIA lines for tracking all by themselves...

A netmail that passes my system has a via line stamp from my tracker
nothing else, as the tracker moves the netmails from the netmail folder
directly into the BSO .... no tosser, no mailer via line, despite the message
is an areafix netmail response ....


ml> i think maybe the only useful function i get from a netmail tracker is
ml> bouncing of unwanted netmail but it has been so long since i've needed
ml> that that i don't recall...

I have 66 individual netmail routing statements ...
from individual points, to individual nodes, to several individual routings
into different nets, individual routings into different regions and
individual routings to other zones ... 66 individual routing statements in
total ... for "normal" routing ...

followed by 14 individual file routings,
followed by 30 individual direct routings
followed by 18 individual hold/direct routings
followed by 15 individual crash/direct routings

For Itrack I've programmed a VoteMgr system, sysop point netmail handling,
complex point reroutings, a pointlist segments receiving and extraction
system from netmails.
What I still hadn't programmed is a programmable percolator :D

[Ulrich Schroeter]

To: Mark Lewis
Hi Mark,

Saturday September 28 2013 16:21, you wrote to me:

ml> On Fri, 27 Sep 2013, Ulrich Schroeter wrote to G00r00:

US>> As Tracker I have Itrack running.
US>> Itrack is a modular construction kit, similar a development
US>> framework. you program scripts for checking, selection, bundling,
US>> transfering to outbound one of the trackers task is dupe

US>> detection caused by circular paths. The simple fix, check the via
US>> kludges. Most of the netmails playing ping-pong between two
US>> systems can be identified with this method. But this method
US>> doesn't work, if the circular path has a hop in between.

ml> hunh? you cannot detect a circular VIA path that has more than two
ml> systems in it? you should be able to easily detect a circular path
ml> with a hundred systems in it... it isn't that hard to do...

via lines analysis is a bit tricky ... as each line requires individual checks
(if not the last line)
looping through netmail lines without a definite end marker is bit tricky with
the construction kit that uses macros to identify kludge lines %MSGID% or
%REPLY% extracts the related msgid or reply kludge lines. and the %LASTVIA%
extracts the last via line .. to identify others then the last via line
and LOOP, LOOP2 or LOOP3 that means "this mail contains one/two/three itrack
vialine of this system in front of others via's"
otherwise it requires individual programming similar to the following example:

Example :
Notify all systems in the mail path of a loop mail.

VARIABLE VIANODECOUNTER 1
SELECTTO @Attribute LOOP2
COPYAREA NETMAIL
FORWARD LOOPTPL
CLEARATTRIBUTES
SETATTRIBUTE @ATTRIBUTE INVALIDVIA
WHILETO !@ADDRESS 0:0/0.0
CHANGETO ADDRESS %VIANODE[%VIANODECOUNTER%]%
SELECTO !SYSTEM &!@ADDRESS 0:0/0.0
COPYAREA NETMAIL
#END# COPYAREA
#END# SELECTTO
SETVARIABLE VIANODECOUNTER
%EVAL[%VIANODECOUNTER%,+,1]%
#END# WHILETO
#END# COPYAREA
#END# COPYAREA
#END# SELECTTO

[Ulrich Schroeter]

To: Mark Lewis
Hi Mark,

Saturday September 28 2013 16:27, you wrote to me:

ml>> Warning, Will Robinson! Warning!

ml>> CRCs are easily duped with different data... additionally, you
ml>> cannot guarantee no repeats in a three year period...

ml>> using a CRC16 or CRC32 for the serialno of a FTN message is a

ml>> very very very B.A.D. idea... i haven't heard of CRC64 yet...
ml>> perhaps it has a larger "key space" that would result in less
ml>> possible duplicates but already MD5 is found to be able to be
ml>> duplicated with different data...

US>> fts-0009 only describes the syntax
US>> ^MSGID: origaddr serialno
US>> but not how the serialno has to be constructed

ml> true...

US>> There is a good description in fidonews FIDO711.NWS (12 Mar 1990)
US>> from jim nutt

ml> yeah... kinda and not...

US>> ^MSGID: zone:net/node.point@domain serialno

US>> It is not the intent to limit the address field of the

US>> msgid to fidonet style addresses, they are used
US>> here simply for illustration, messages originating from
US>> other types of systems should use an address appropriate to
US>> the originating system. The serial number may be
US>> anything the developer's little heart desires, AS LONG AS
US>> IT IS UNIQUE, NO TWO MESSAGES ON A SYSTEM MAY SHARE A SERIAL
US>> NUMBER!!

ml> not /on/ a system but _from_ a system... the entire control line
ml> contents are used for the detection... address and serial number...

depends on the viewpoint ...
from the PoV of your own system its /on/
from the PoV of another system its /from/
the result is the same ... as the combination system+serial#
is the measure that counts

US>> The serial number is formatted as an 8 character

US>> hexadecimal integer, i.e. a 32 bit word. since this
US>> yields in the neighborhood of 4 billion unique numbers, I don't
US>> think it will be a limit for most systems. A common choice for
US>> the serial number is the number of seconds since 1 jan 1970

US>> 00:00:00 gmt, this is unique on a single user system and
US>> relatively easy to generate.

ml> this limits a system to only one message per second or maybe one
ml> message per two seconds... much too slow...

US>> A more elaborate system will doubtless be necessary in

US>> the case of a multi-user system.

ml> there is that, too...

ml> however, i have posting software that easily generates several hundred
ml> messages per second... with no duplicated serial numbers... so, yeah,
ml> as noted above, kinda and not...

ml> there is also a recent proposal in the FTSC library concerning the
ml> serial number generation... it is close to what i use but what i use
ml> was written back in the '90s and is also multinode (bbs node)
ml> usable...

ml> but then again, one doesn't really need anything fancy to use a simply
ml> incrementing counter that starts at zero, do they? ;)

if you have a deeper look into the reply chain of earlier post (if it wasn't my
first reply) you'll find my message where I still have posted this as one of
the possible options

[mark lewis]

To: Ulrich Schroeter

On Sun, 29 Sep 2013, Ulrich Schroeter wrote to Mark Lewis:

US>> so no special "ZONEGATE" kludge is placed into the netmail ...
US>> but the msgheader address is set to zonegate aka 2/1

ml> i never implied that there was any kind of zongate related control
ml> line...

US> from the original mail I've responded to:

[trim]

the (other) control line i was speaking of is the MSGID... there may also be
TOPT and FMPT control lines to be consulted and possibly even one or two
others...

[Nicholas Boel]

To: Ulrich Schroeter
US> For Itrack I've programmed a VoteMgr system, sysop point netmail
US> handling, complex point reroutings, a pointlist segments receiving and
US> extraction system from netmails.
US> What I still hadn't programmed is a programmable percolator :D

I think I'm beginning to understand correctly here. If any software you
didn't program doesn't work /exactly/ the way your software does, you're
going to complain to the developer until they change it to match?

You don't even use the software. James has only started adding FTN stuff to
it quite recently. Let him do what he's doing and if there's any severe
problems on the way, let him know.. rather than trying to force your ideas.

Regards,
Nick

--- Mystic BBS v1.10 A38 (Linux)
* Origin: Dark Sorrow | telnet://bbs.darksorrow.us (1:154/701)

[Ulrich Schroeter]

To: Nicholas Boel
Hi Nicholas,

Monday September 30 2013 11:01, you wrote to me:

US>> For Itrack I've programmed a VoteMgr system, sysop point netmail
US>> handling, complex point reroutings, a pointlist segments

US>> receiving and extraction system from netmails. What I still
US>> hadn't programmed is a programmable percolator :D

NB> I think I'm beginning to understand correctly here. If any software
NB> you didn't program doesn't work /exactly/ the way your software does,
NB> you're going to complain to the developer until they change it to
NB> match?

NB> You don't even use the software. James has only started adding FTN
NB> stuff to it quite recently. Let him do what he's doing and if there's
NB> any severe problems on the way, let him know.. rather than trying to
NB> force your ideas.

my tosser has currently 52 active links configured.
From all these links I have no problem with all their sent netmail and echomail
bundles.
before tossing all mails are backed up, day by day, on hold for 7 days
for recovery and debugging purposes.
no problems for 50 links, except the two links using mystic

Rudi has sent today in the morning a couple of netmails, only one packet
remains that I can analyse 'cause all packets named identical
00f40460.pkt ... thats my nodenumber in hex => FAIL by sender

same with the echomail bundles that delivers the identical
filename all the day long :-P


NB> Regards,
NB> Nick

[Nicholas Boel]

To: Ulrich Schroeter
US> my tosser has currently 52 active links configured.

I didn't know this had turned into an e-peen war. May I bow to your Fidonet
awesomeness. Or not. I could care less how many active links you have
configured. As a matter of fact, I probably have that many links JUST in the
'othernet' I run. So what? That means nothing here.

US> From all these links I have no problem with all their sent netmail and
US> echomail bundles.

Bummer. So you finally found one, and you decide to rag on the programmer
in quite a rude, forceful matter. Good luck getting anything accomplished
that way, chief.

US> no problems for 50 links, except the two links using mystic

Do realize those two links using Mystic are the newest additions to using
Mystic as well. They still have quite a bit to learn with the software. On
the other hand, did you notice their bug reports are being tended to, since
they're being rather nice about it. Whereas you're trying to tell the
developer how it "should" be (in your eyes and the proposals that are rotting
in the depths of the FTSC). I wouldn't be surprised if the developer is
ignoring most of your messages, as you've been quite rude while not even
using the software. *shrug*

[mark lewis]

To: Ulrich Schroeter

On Mon, 30 Sep 2013, Ulrich Schroeter wrote to Mark Lewis:

US>> again, please split netmail handling and echomail handling. This
US>> are two seperated fields of FTN activity

US>> Tossers handles echomails, Trackers handles netmails

ml> not over here they do not... trackers track and nothing else...
ml> tossers toss the mail into and scan it out of the bbs' message
ml> bases...

US> tossers are dumb silly echomail shufflers ...

i highly disagree... no offense intended but your experience seems to be
limited... IMail, FMail, GEcho, Fastecho and others all handle netmail with no
problems whatsoever... i started with IMail when it first came out back in the
'80s... it is the tosser that allowed our RemoteAccess system to properly
process FTN mail when the QuickBBS mail tools would not work properly... then i
checked out FMail some years later when it came out... tried GEcho for a short
time, too... settled on Fastecho and was asked to join the beta team for it...
currently i use FMail on a few point systems...

US> they are optimized for. most tossers also supports netmail
US> packaging, but this isn't the tossers main area and on complex
US> netmail handling they'll fail so tossers are mostly included in
US> point packets

what "complex handling" are you talking about? there is none related to
netmail... handling netmail is easier than handling echomail...

US> mail exchangers (hosts, and probably also echomail hubs) using
US> trackers for netmail processing 'cause they can support node and
US> pointlists (that tossers seldom support)

US> crashmail, routed mail, nodelist lookups, netmail bouncing, host-
US> and hub-routing, subgroup routing ... dynamic outbound queue
US> mailers like frontdoor routes mails according to routing configs
US> and events. binkley + binkd with a static outbound requires
US> another tool that handles netmail routing. This task is
US> accomplished by Trackers.

the mail tossers handle this task just fine for the BSO systems i have set
up... no trackers needed... mail tossers that i'm familiar with use the same or
very similar routing language that frontdoor uses...

ml> my ot-track only tracks and msgtrack before it only tracks... but
ml> then, i really don't need either since my mailer and tosser add

ml> the VIA lines for tracking all by themselves...

US> A netmail that passes my system has a via line stamp from my
US> tracker nothing else, as the tracker moves the netmails from the
US> netmail folder directly into the BSO .... no tosser, no mailer via
US> line, despite the message is an areafix netmail response ....

yeah, none of the trackers i use has ever been used to pack netmail... not in
BSO or dynamic mailer mode... it wasn't ever seen as necessary because the
tosser handled it... the only thing that a tracker give me that a tosser
doesn't is a log if each netmail's particulars... nothing else... i've never
used the netmail bounce features for netmail from undesirable systems and i've
never used the "vacation reply" stuff... i don't like that vacation stuff
anyway... it is annoying in email, too ;)

ml> i think maybe the only useful function i get from a netmail

ml> tracker is bouncing of unwanted netmail but it has been so long
ml> since i've needed that that i don't recall...

US> I have 66 individual netmail routing statements ...

interesting... i have no where near that and don't need any more than what i
have to route netmail, echomail and files attached to netmail ;)

US> from individual points, to individual nodes, to several individual
US> routings into different nets, individual routings into different
US> regions and individual routings to other zones ... 66 individual
US> routing statements in total ... for "normal" routing ...

US> followed by 14 individual file routings,
US> followed by 30 individual direct routings
US> followed by 18 individual hold/direct routings
US> followed by 15 individual crash/direct routings

wow... much too much... hehehe ;)

US> For Itrack I've programmed a VoteMgr system,

interesting... i do vote tallying either by hand or by using the votemgr
program... either works just as well as the other ;)

US> sysop point netmail handling, complex point reroutings, a
US> pointlist segments receiving and extraction system from netmails.
US> What I still hadn't programmed is a programmable percolator :D

hahaha... i hear ya there! i'm still waiting on others to provide the same but
i can do something with their schedulers and a relay device connected to a
serial or parallel port to switch on power to an outlet at a certain time and
switch it off again later ;) ;) ;)

[mark lewis]

To: Ulrich Schroeter

On Mon, 30 Sep 2013, Ulrich Schroeter wrote to Mark Lewis:

ml> hunh? you cannot detect a circular VIA path that has more than two
ml> systems in it? you should be able to easily detect a circular path
ml> with a hundred systems in it... it isn't that hard to do...

US> via lines analysis is a bit tricky ...

what is tricky about starting at the topmost VIA line and going down thru them
recording the VIA creation software and the VIA line creation address? you're
only looking for duplicate lines separated by at least one other VIA line from
another system... nothing difficult about that ;)

US> as each line requires individual checks (if not the last line)
US> looping through netmail lines without a definite end marker

hunh? there are several ways to detect the end of the message... #00 null
bytes, EOF or quite simply, the end of the data... nothing so hard there,
either...

US> is bit tricky with the construction kit that uses macros to
US> identify kludge lines %MSGID% or %REPLY% extracts the related
US> msgid or reply kludge lines. and the %LASTVIA% extracts the last
US> via line .. to identify others then the last via line and LOOP,
US> LOOP2 or LOOP3 that means "this mail contains one/two/three itrack
US> vialine of this system in front of others via's"

wow [laughing] either i'm not understanding or this has been made more
difficult than it needs to be ;)

US> otherwise it requires individual programming similar to the
US> following example:

US> Example :
US> Notify all systems in the mail path of a loop mail.

US> VARIABLE VIANODECOUNTER 1
US> SELECTTO @Attribute LOOP2
US> COPYAREA NETMAIL
US> FORWARD LOOPTPL
US> CLEARATTRIBUTES
US> SETATTRIBUTE @ATTRIBUTE INVALIDVIA
US> WHILETO !@ADDRESS 0:0/0.0
US> CHANGETO ADDRESS %VIANODE[%VIANODECOUNTER%]%
US> SELECTO !SYSTEM &!@ADDRESS 0:0/0.0
US> COPYAREA NETMAIL
US> #END# COPYAREA
US> #END# SELECTTO
US> SETVARIABLE VIANODECOUNTER
US> %EVAL[%VIANODECOUNTER%,+,1]%
US> #END# WHILETO
US> #END# COPYAREA
US> #END# COPYAREA
US> #END# SELECTTO

interesting...

[mark lewis]

To: Ulrich Schroeter

On Mon, 30 Sep 2013, Ulrich Schroeter wrote to Mark Lewis:

US>> It is not the intent to limit the address field of the

US>> msgid to fidonet style addresses, they are used here
US>> simply for illustration, messages originating from other
US>> types of systems should use an address appropriate to the
US>> originating system. The serial number may be anything
US>> the developer's little heart desires, AS LONG AS IT IS
US>> UNIQUE, NO TWO MESSAGES ON A SYSTEM MAY SHARE A SERIAL

US>> NUMBER!!

ml> not /on/ a system but _from_ a system... the entire control line
ml> contents are used for the detection... address and serial number...

US> depends on the viewpoint ...

sure if one wants to complicate things... in the simple way, it is a message...
no matter where it originates from, the MSGID contents are from that system...

US> from the PoV of your own system its /on/
US> from the PoV of another system its /from/
US> the result is the same ...

so why complicate the explanation or definition? ;)

US> as the combination system+serial# is the measure that counts

right...

[Ulrich Schroeter]

To: Nicholas Boel
Hi Nicholas,

Monday September 30 2013 23:36, you wrote to me:

US>> my tosser has currently 52 active links configured.

NB> I didn't know this had turned into an e-peen war. May I bow to your
NB> Fidonet awesomeness. Or not. I could care less how many active links
NB> you have configured. As a matter of fact, I probably have that many
NB> links JUST in the 'othernet' I run. So what? That means nothing here.

then you should know how unified arcmail bundles receives your system (probably
or not)

US>> From all these links I have no problem with all their sent

US>> netmail and echomail bundles.

NB> Bummer. So you finally found one, and you decide to rag on the
NB> programmer in quite a rude, forceful matter. Good luck getting
NB> anything accomplished that way, chief.

if it feels rude on the receivers side, I'm sorry ...
this isn't my intension

what I want to accomplish is a "Let exercise more caution on FTN programming"
a one-dimensional programming within the bbs package may work, but programming
an ftn interface requires a quite more look outside the box what may happen to
others

US>> no problems for 50 links, except the two links using mystic

NB> Do realize those two links using Mystic are the newest additions to
NB> using Mystic as well. They still have quite a bit to learn with the
NB> software. On the other hand, did you notice their bug reports are
NB> being tended to, since they're being rather nice about it. Whereas
NB> you're trying to tell the developer how it "should" be (in your eyes
NB> and the proposals that are rotting in the depths of the FTSC). I
NB> wouldn't be surprised if the developer is ignoring most of your
NB> messages, as you've been quite rude while not even using the software.
NB> *shrug*

currently I'm not a software user, but an affected uplink node, affected by two
downlinks using this software as their main system/tosser
maybe one day, if the ftn interface is complete, I'll put it in my work queue
to install an addtl. Telnet BBS, but currently I'm still busy with some other
projects that I have to finish first before I start a new project
as said before, if my reports and proposals sounds quite rude, than its
probably also to do with the long term experience receiving content from an
unknown count of links with an also unknown count of different software
at linkpartners side that didn't cause any problems before that now two new
Mystic downlinks causes on my system while delivering non-unique mailfiles


NB> Regards,
NB> Nick

NB> -$- Mystic BBS v1.10 A38 (Linux)
NB> $ Origin: Dark Sorrow | telnet://bbs.darksorrow.us (1:154/701)

[Nicholas Boel]

To: Ulrich Schroeter
US> currently I'm not a software user, but an affected uplink node, affected
US> by two downlinks using this software as their main system/tosser
US> maybe one day, if the ftn interface is complete, I'll put it in my work
US> queue to install an addtl. Telnet BBS, but currently I'm still busy with
US> some other projects that I have to finish first before I start a new
US> project as said before, if my reports and proposals sounds quite rude,
US> than its probably also to do with the long term experience receiving
US> content from an unknown count of links with an also unknown count of
US> different software at linkpartners side that didn't cause any problems
US> before that now two new Mystic downlinks causes on my system while
US> delivering non-unique mailfiles

The question here is, are anyone else in this echo complaining about how it
works? There's a lot more uplinks than just you that Mystic systems are
sending their mail to.

As I said before, maybe it's YOUR software that's wrong. Instead of coming
around here telling the Mystic developer that he's wrong and he needs to fix
these things.. maybe you should take another look at your own software. Just
sayin'.

Regards,
Nick

--- Mystic BBS v1.10 A38 (Linux)
* Origin: Dark Sorrow | telnet://bbs.darksorrow.us (1:154/701)

[Ulrich Schroeter]

To: Mark Lewis
Hi Mark,

Wednesday October 02 2013 11:21, you wrote to me:

ml> On Mon, 30 Sep 2013, Ulrich Schroeter wrote to Mark Lewis:
US>>> again, please split netmail handling and echomail handling. This
US>>> are two seperated fields of FTN activity
US>>> Tossers handles echomails, Trackers handles netmails

ml>> not over here they do not... trackers track and nothing else...
ml>> tossers toss the mail into and scan it out of the bbs' message
ml>> bases...

US>> tossers are dumb silly echomail shufflers ...

ml> i highly disagree... no offense intended but your experience seems to
ml> be limited... IMail, FMail, GEcho, Fastecho and others all handle
ml> netmail with no problems whatsoever... i started with IMail when it
ml> first came out back in the '80s... it is the tosser that allowed our
ml> RemoteAccess system to properly process FTN mail when the QuickBBS
ml> mail tools would not work properly... then i checked out FMail some
ml> years later when it came out... tried GEcho for a short time, too...
ml> settled on Fastecho and was asked to join the beta team for it...
ml> currently i use FMail on a few point systems...

ok other question:

what does have echomail distribution to do with netmail routing ?

netmail routing follows fidonet structures by default (exceptions possible)
echomail distribution goes it own ways and doesn't require any fidonet
structures (aka ZC's,RC's,NC's). The combination of echomail distribution and
netmail forwarding combines at the moment you try to answer to an echomail with
a unique netmail reply to a specific recipient.

sample:
A Node1 (A) from RC2,Net2 receives an echomail from Node2 (B) from RC1,Net2
via Echomail Hub2 via Echomail Hub3 and starts a netmail reply.
The netmail reply doesn't follow the echomail distribution structure
but follows the netmail routing default structure, that goes to Net2 Host
of RC2. In the case Net2 Host sends Host routed the netmail will be forwarded
to RC1, Net2 Host who delivers the netmail to Node2 of Net2 of RC1
Netmail Routing is handled under P407, Echomail distribution not.
More, echomail distribution is highly encouraged not to combine onto a net host
system (ok this is all oldschool theory :) ..... status 1989 as P407 was
written :D ...) but also today often Echomail Hubs are often systems that
aren't related specific to a Net Host or RC system .....

Netmail Forwarding Schema Echo Distribution Schema
(top-down schema) (ring/star/concatenated structures)

ZC
|
+-- RC1
| |
| | Echomail Hub1 <=+
| +-Net1 | | | |
| | +- Node1 --------------------+ | | |
| | +- Node2 ---------------------+ | | |
| | | | | |
| +-Net2 | | | |
| | +- Node1 -----------------------+ | |
| |[B] +- Node2 -------------------------+ | |
| | +- Node3 -------------------------|-+ |
| | | | |
| +- Net3 Echomail Hub2 <=+
| | | | |
| +- Node1 ---------------------+ | |
| +- Node2 ---------------------+ | |
+-- RC2 | | |
| | | |
+- Net1 | | |
| +- Node1 -----------------------+ |
| +- Node2 ---------------------|-+ |
| | | |
+- Net2 Echomail Hub3 <=+
[A] +- Node1 ---------------------+ |
+- Node2 -----------------------+

The default netmail routing structures are similar to the old days netmail
routing following host to host routing or netmail forwarding to known transfer
systems (Zonegates, inter-regional hosts). The days where costs did count much,
often netmail routing was used. Today with a majority of IP to IP links with no
extra costs diffuses a netmail routing structure as also individual nodes
started netmail crashing to the final destination node.

Since about 10 years, starting Zone2 pointlist distribution I have direct links
into mostly all regions within Zone2 so I've also started individual netmail
routing to all the linked regions. With other special stuff distribution more
and more individual links started, with same individual netmail routings.
So there is also a special indirect Netmail routing and echomail routing via
Region34 for the Spanish/Portuguese community. In the R34 sample the netmail
routing follows the echomail distribution path. But this is an exception to all
the other Netmail routing paths and echomail distribution paths .. eg.
international echomail distribution goes zone1 - zone3 - zone2 where netmail
goes direct zone2 - zone1.

Currently we encourage also the end nodes and points to have at least two
different uplinks, the reason given, in case one uplink disappears, the 2nd
link can be easily activated or if one echo isn't available at one uplink, it
can be requested from a 2nd uplink.

Today a downlink discovered a broken link for an international echo for about 6
days. Once notified I've today switched from the default uplink for
international echoes to a more direct link. This doesn't affects any netmail
routing ...

Echomail business is a totaly separated business from netmail business (FTN
structural business).
From an end nodes PoV this doesn't seems to be that easyly to discover.

For netmail forwarding you more have to know about the netmail routing
structure that one gets out of a nodelist (routing tables, routing definitions)
in contrast to a link table of echomail links for each individual echomail
area.

So what you have to take care off while exporting echomails is to dumb export
mails multiple times .. to each linked connection system for a specific
echoarea. You don't have to take much care about ftn addressing while exporting
echomails .. there are only 2 aka's that counts .. the own aka and the link
partners aka - individual for each echoarea.
Otherwise in netmail handling ....
do you have a default netmail routing ? is the netmail flagged crash or direct?
then you have to seperate netmail delivery destination system from the default
uplink system (different destination aka's) while exporting a netmail.
For the configured netmail uplink there maybe a packet password defined, this
doesn't count for the crash netmail delivery to an unconfigured netmail
recipient. For a direct delivery of crashmails, you addtl. have to know the
direct addressing - phone number to dial, IP address for delivery via IP
Then you also have to take care about the status of the destination node. On
status Hold or Down, netmail forwarding has to stop and netmail has to be set
on hold until the status will change one day to prevent any problems about P407
2.1.7 Not Routing Mail
[...]
2.1.7 Not Routing Mail (2nd block)
[...]
"If you do not forward a message when you previously agreed to perform such
routing, the message must be returned to the sysop of the node at which it
entered FidoNet with an explanation of why it was not forwarded. (It is not
necessary to return messages which are addressed to a node which is not in
the current nodelist.) Intentionally stopping an in-transit message without
following this procedure constitutes annoying behavior. In the case of a
failure to forward traffic due to a technical problem, it does not become
annoying unless it persists after being pointed out to the sysop."

so the recommendation is to start a nodelist lookup about the destinations aka
status before sending out a netmail to fidonet's world ....

ml> )\/(ark
ml> $ Origin: (1:3634/12)

[Ulrich Schroeter]

To: Mark Lewis
Hi Mark,

Wednesday October 02 2013 11:38, you wrote to me:

ml> On Mon, 30 Sep 2013, Ulrich Schroeter wrote to Mark Lewis:
ml>> hunh? you cannot detect a circular VIA path that has more than

ml>> two systems in it? you should be able to easily detect a circular
ml>> path with a hundred systems in it... it isn't that hard to do...

US>> via lines analysis is a bit tricky ...

ml> what is tricky about starting at the topmost VIA line and going down
ml> thru them recording the VIA creation software and the VIA line
ml> creation address? you're only looking for duplicate lines separated by
ml> at least one other VIA line from another system... nothing difficult
ml> about that ;)

the built-in macros has only loop, loop2 and loop3 available, that means

via5 ?
via4 ? no more loop macro .. program by your own :/
via3 thats me? Select Loop3 then do something
via2 ^loop3 thats me? Select Loop2 then do something
via1 ^loop2 thats me? Select Loop then do something
^loop

US>> as each line requires individual checks (if not the last line)
US>> looping through netmail lines without a definite end marker

ml> hunh? there are several ways to detect the end of the message... #00
ml> null bytes, EOF or quite simply, the end of the data... nothing so
ml> hard there, either...

US>> is bit tricky with the construction kit that uses macros to
US>> identify kludge lines %MSGID% or %REPLY% extracts the related
US>> msgid or reply kludge lines. and the %LASTVIA% extracts the last
US>> via line .. to identify others then the last via line and LOOP,
US>> LOOP2 or LOOP3 that means "this mail contains one/two/three

US>> itrack vialine of this system in front of others via's"

ml> wow [laughing] either i'm not understanding or this has been made more
ml> difficult than it needs to be ;)

how many parts can have a mail? all specific parts you can select with a
keyword selection. This applys for seenbys, this also applys for path lines.
but not for via lines :-P ... its limited upto a 3 via lines back
Selecting message body lines (excluding kludge lines, PID, MSGID, REPLY, INTL
and all that stuff can be selected by macro keywords) has a complex syntax as
its seldom required to extract content lines from somewhere the middle or
bottom of a message. Selecting line 1 of a message is still easy line[%1] or
similar syntax as you can hardcode line "1" ... but with an unknown count of
lines _this_ becomes tricky as there is no script parameter available to count
the number of lines in a message. So as an example to truncate lines below a
tearline in an echomail message you have to add your own program routines:
first to scroll thru the whole message to get the maximum of lines. Then
you have to program a 2nd routine of individual backwards line selections, line
by line until you get a hit for what you're looking for eg the tearline

an analogy I probably can give with a sql database handling by sql queries ...
trying to extract an individual line of a text blob column -> with a sql select
statement :D
names, subject, datetime, akas, individual kludge lines (except via lines) all
similiar to a sql column and the message body as one combined text blob ...

US>> otherwise it requires individual programming similar to the
US>> following example:

US>> Example :
US>> Notify all systems in the mail path of a loop mail.

US>> VARIABLE VIANODECOUNTER 1
US>> SELECTTO @Attribute LOOP2
US>> COPYAREA NETMAIL
US>> FORWARD LOOPTPL
US>> CLEARATTRIBUTES
US>> SETATTRIBUTE @ATTRIBUTE INVALIDVIA
US>> WHILETO !@ADDRESS 0:0/0.0
US>> CHANGETO ADDRESS %VIANODE[%VIANODECOUNTER%]%
US>> SELECTO !SYSTEM &!@ADDRESS 0:0/0.0
US>> COPYAREA NETMAIL
US>> #END# COPYAREA
US>> #END# SELECTTO
US>> SETVARIABLE VIANODECOUNTER
US>> %EVAL[%VIANODECOUNTER%,+,1]%
US>> #END# WHILETO
US>> #END# COPYAREA
US>> #END# COPYAREA
US>> #END# SELECTTO

ml> interesting...

as above ... there are all individual keywords for each ftn message specific
parameters ... and a lot of what you can execute on messages ... all
parameters ... except vialines that is limited to a maximum of the last 3
vialines :(

[Janis Kracht]

To: Ulrich Schroeter
Hello Uli,

> ok other question:

> what does have echomail distribution to do with netmail routing ?

This doesn't have to do with Mystic, but just a note.. over here in Zone 1,
echomail routed netmail often the norm. i.e, if you know someone's uplink for
echomail is 1:261/38, you can route netmail to that link here. Any of my
downlinks can generally get echomail routed to them, unless they tell me they
do not want to pick up any netmail here. If they tell me to not route netmail
to them this way, I can tell BBBS to not do this.

Take care,
Janis

--- BBBS/Li6 v4.10 Dada-1
* Origin: Prism bbs (1:261/38)

[mark lewis]

To: Ulrich Schroeter

On Tue, 01 Oct 2013, Ulrich Schroeter wrote to Nicholas Boel:

US> my tosser has currently 52 active links configured. From all these
US> links I have no problem with all their sent netmail and echomail
US> bundles. before tossing all mails are backed up, day by day, on
US> hold for 7 days for recovery and debugging purposes. no problems
US> for 50 links, except the two links using mystic

your archiving methods do not account for stoneage PKTs and bundles... most
likely you are simply copying the files to another directory before
processing... i do the same... to handle stoneage traffic means that we must
look deeper into the files and rename them on our own...

FWIW: this has been the case since BSO was invented... it has only been in
'recent' years that this was seen as a problem and other methods were developed
to work around the problem...

james has just recently added FTN capabilities to mystic... he is starting at
the beginning and currently limiting his work to that of the BSO... as such, he
has only the available BSO documentation to go by... i have pointed out several
times in the past that other naming techniques may be used as long as the basic
?LO and ?UT methods are used... only those files need follow the XXXXYYYY
naming where XXXX is the destination net in HEX and YYYY is the destination
node in HEX... the ?UT can be renamed to anything the mailer chooses when its
name is sent to the remote system... at that point, it could easily be the
current elapsed number of seconds since 1970 in HEX or it could be anything
else...

there is almost the same situation with bundles but in any case, the
documentation and de facto(??) standard that james is following is that
specified in the original binkleyterm documentation... at some point, things
will likely change but until then, we must remember that the software is in
alpha stage... technically speaking, many would think that that would limit the
software's usage to a small dedicated team of testers and the author... beta
would be open to a larger team of testers and at some point, public release
would be achieved when the software reached a definite stable and operational
point...

speaking historically, mystic's current status and methodology would NOT cause
it to be banned from use in fidonet as other software (binkleyterm and
frontdoor) were banned during development stages until fixed and tested...

[mark lewis]

To: Ulrich Schroeter

On Thu, 03 Oct 2013, Ulrich Schroeter wrote to Nicholas Boel:

US>> my tosser has currently 52 active links configured.

NB> I didn't know this had turned into an e-peen war. May I bow to

NB> your Fidonet awesomeness. Or not. I could care less how many
NB> active links you have configured. As a matter of fact, I probably
NB> have that many links JUST in the 'othernet' I run. So what? That
NB> means nothing here.

US> then you should know how unified arcmail bundles receives your
US> system (probably or not)

there are many system operators that don't know and don't care /how/ it all
works as long as it works for them ;)

US>> From all these links I have no problem with all their sent
US>> netmail and echomail bundles.

NB> Bummer. So you finally found one, and you decide to rag on the
NB> programmer in quite a rude, forceful matter. Good luck getting
NB> anything accomplished that way, chief.

US> if it feels rude on the receivers side, I'm sorry ... this isn't
US> my intension

+100

US> what I want to accomplish is a "Let exercise more caution on FTN
US> programming" a one-dimensional programming within the bbs package
US> may work, but programming an ftn interface requires a quite more
US> look outside the box what may happen to others

this can only be done when one experiances it first hand or others can describe
it in easily understood language and concepts... sometimes "we", yes i've been
known to fall into it, too, get wrapped up in the details and can't get the
concept or ideas acrossed clearly...

US>> no problems for 50 links, except the two links using mystic

NB> Do realize those two links using Mystic are the newest additions

NB> to using Mystic as well. They still have quite a bit to learn with
NB> the software.

to nick: while that may be, the software should not allow one to shoot one's
own foot or allow one to unknowingly cause problems on other systems...

NB> On the other hand, did you notice their bug reports are being
NB> tended to, since they're being rather nice about it. Whereas

NB> you're trying to tell the developer how it "should" be (in your

NB> eyes and the proposals that are rotting in the depths of the
NB> FTSC). I wouldn't be surprised if the developer is ignoring most
NB> of your messages, as you've been quite rude while not even using
NB> the software. *shrug*

US> currently I'm not a software user, but an affected uplink node,
US> affected by two downlinks using this software as their main
US> system/tosser maybe one day, if the ftn interface is complete,
US> I'll put it in my work queue to install an addtl. Telnet BBS, but
US> currently I'm still busy with some other projects that I have to
US> finish first before I start a new project

yeah... there's not enough time even in 48 hour days to get everything done...
i learned that long ago when i was living 48 hour days with 12 hour sleep
periods... it was especially tough with the paying job when those with the
checkbook didn't realise how much time i was really spending on their
projects... needless to say, i don't do that any more and most of them are no
longer in business since i left them ;)

US> as said before, if my reports and proposals sounds quite rude,
US> than its probably also to do with the long term experience
US> receiving content from an unknown count of links with an also
US> unknown count of different software at linkpartners side that
US> didn't cause any problems before that now two new Mystic downlinks
US> causes on my system while delivering non-unique mailfiles

there may also be other barriers that "we" are not aware of... i make special
note of some who are either using translators from their native language or are
trying to converse in english which is not their 3rd or even 4th language...
you should see some of the stuff we get in the support forums that i contribute
to... many is the time when we have to tell them to just write in their native
language and we will translate on our own or someone that speaks their native
language will help if there is anyone that can...

[mark lewis]

To: Nicholas Boel

US> currently I'm not a software user, but an affected uplink node,

US> affected by two downlinks using this software as their main
US> system/tosser maybe one day, if the ftn interface is complete,
US> I'll put it in my work queue to install an addtl. Telnet BBS, but
US> currently I'm still busy with some other projects that I have to
US> finish first before I start a new project as said before, if my
US> reports and proposals sounds quite rude, than its probably also to
US> do with the long term experience receiving content from an unknown
US> count of links with an also unknown count of different software at
US> linkpartners side that didn't cause any problems before that now
US> two new Mystic downlinks causes on my system while delivering
US> non-unique mailfiles

NB> The question here is, are anyone else in this echo complaining
NB> about how it works? There's a lot more uplinks than just you that
NB> Mystic systems are sending their mail to.

it is quite possible that other systems simply have not seen the problem and
are completely unaware that they may be losing mail...

NB> As I said before, maybe it's YOUR software that's wrong.

and it may not be his software but any BAT or shell scripts that he may have
employed...

NB> Instead of coming around here telling the Mystic developer that
NB> he's wrong and he needs to fix these things.. maybe you should
NB> take another look at your own software. Just sayin'.

i don't see where anyone has said that james is doing it wrong... only that the
current method can and apparently has lead to the possible loss of mail on
other systems... in US' case, it was seen in the archiving portion of his
processing... i can see that my processing will also lose mail when it archives
pkts and bundles before processing them... but for the most part, the archiving
for debugging purposes is not critical... it does, however, show that the
problem may appear in other places and that's not good...

[mark lewis]

To: Ulrich Schroeter

On Fri, 04 Oct 2013, Ulrich Schroeter wrote to Mark Lewis:

[trim]

US> ok other question:

US> what does have echomail distribution to do with netmail routing ?

that depends...

US> netmail routing follows fidonet structures by default

default netmail routing in fidonet is HOST routed... that means that your
systems send outbound netmail to the HOST of the destination system... HOST
routing is NOT, as many believe it to be, where you send your netmail to your
HOST (NC) which sends it to their HOST (RC) which sends it to their HOST (ZC)
which send it down the line to the RC of the region where the destination
system is which then sends it to the NC of the destination system's net which
then sends it to the destination system... HOST routing is two hops only... the
originating system to the destinaion system's HOST (or HUB as listed in the
nodelist) is one hop... then from the HOST (or HUB) to the destination system
is the second hop... anything other routing scheme is not covered by policy or
technical specs or proposals...

US> (exceptions possible) echomail distribution goes it own ways and
US> doesn't require any fidonet structures (aka ZC's,RC's,NC's).

echomail routing is just another scheme which i include in the above last
statement ;)

US> The combination of echomail distribution and netmail forwarding
US> combines at the moment you try to answer to an echomail with a
US> unique netmail reply to a specific recipient.

yeah... and? i'll read on and see if i can see what you are trying to convey...

US> sample:
US> A Node1 (A) from RC2,Net2 receives an echomail from Node2 (B) from
US> RC1,Net2 via Echomail Hub2 via Echomail Hub3 and starts a netmail
US> reply. The netmail reply doesn't follow the echomail distribution
US> structure but follows the netmail routing default structure, that
US> goes to Net2 Host of RC2. In the case Net2 Host sends Host routed
US> the netmail will be forwarded to RC1, Net2 Host who delivers the
US> netmail to Node2 of Net2 of RC1

yeah, this is exactly what i explained above that default fidonet netmail
routing is NOT... there is only one HOST involved in default FTN routing and
covered by fidonet policy... that HOST is the /destination's/ HOST... no other
HOST is involved... remember, the cost of delivering netmail is on the
originator of that netmail unless they have an agreement with another system to
route it for them... since cost is real, then the originator pays for the
delivery to the nearest point of the destination system or directly to the
destination system... that means the destination system's HOST or possibly
their nodelist HUB... HUBs in the nodelist were only for netmail routing and to
ease nodelist segment creation since a HUB was basically a "HOST in training"
of sorts...

US> Netmail Routing is handled under P407, Echomail distribution not.
US> More, echomail distribution is highly encouraged not to combine
US> onto a net host system (ok this is all oldschool theory :)

yep... theory is accurate ;)

US> ..... status 1989 as P407 was written :D ...) but also today often
US> Echomail Hubs are often systems that aren't related specific to a
US> Net Host or RC system .....

right... that is why netmail traveling with echomail is said to be "echomail
routed"... the only other two methods are HOST/HUB routed and direct/crash...

US> Netmail Forwarding Schema Echo Distribution Schema
US> (top-down schema) (ring/star/concatenated
US> structures)

[confusing chart removed]

US> The default netmail routing structures are similar to the old days
US> netmail routing following host to host routing

there is/was no such thing as HOST to HOST routing by default in fidonet...

US> or netmail forwarding to known transfer systems (Zonegates,
US> inter-regional hosts). The days where costs did count much, often
US> netmail routing was used. Today with a majority of IP to IP links
US> with no extra costs diffuses a netmail routing structure as also
US> individual nodes started netmail crashing to the final destination
US> node.

which basically says that routed netmail is not as big a thing today as it was
back then... that would seem to indicate that netmail trackers and routing
stuffs is not needed so much today and doesn't need to be very elaborate at
all...

US> Since about 10 years, starting Zone2 pointlist distribution I have
US> direct links into mostly all regions within Zone2 so I've also
US> started individual netmail routing to all the linked regions.

that's on you and a service that you provide to split routed netmail off to
traverse possibly a shorter distance then if you had sent it on to your next
configured link for it...

US> With other special stuff distribution more and more individual
US> links started, with same individual netmail routings. So there is
US> also a special indirect Netmail routing and echomail routing via
US> Region34 for the Spanish/Portuguese community. In the R34 sample
US> the netmail routing follows the echomail distribution path. But
US> this is an exception to all the other Netmail routing paths and
US> echomail distribution paths .. eg. international echomail
US> distribution goes zone1 - zone3 - zone2

this doesn't have to be this way, though... but i'm not going to wade into the
depths of why it is in play as it is... those who have read FN_SYSOP and/or
FIDONEWS for the last 10 years all know exactly why and i'll leave it right
there...

US> where netmail goes direct zone2 - zone1.

ok... so you (and others) provide a possible shorter path for routed netmail
that passes thru your system... i do the same but i don't purport to call it
what it is not...

US> Currently we encourage also the end nodes and points to have at
US> least two different uplinks, the reason given, in case one uplink
US> disappears, the 2nd link can be easily activated or if one echo
US> isn't available at one uplink, it can be requested from a 2nd
US> uplink.

understood but don't see where it has any bearing on the discussion of netmail
handling and routing...

US> Today a downlink discovered a broken link for an international
US> echo for about 6 days. Once notified I've today switched from the
US> default uplink for international echoes to a more direct link.
US> This doesn't affects any netmail routing ...

why switch feeds for that echo area? why not follow the trail upstream to find
the break and get it fixed? or is politics also a plague over there in Z2 where
things like this just can't be worked out?

US> Echomail business is a totaly separated business from netmail
US> business (FTN structural business).

i cannot agree... there is no FTN structure involved in default netmail routing
(aka HOST routed)...

US> From an end nodes PoV this doesn't seems to be that easyly to
US> discover.

because many are not interested in knowing the details :?

US> For netmail forwarding you more have to know about the netmail
US> routing structure that one gets out of a nodelist (routing tables,
US> routing definitions) in contrast to a link table of echomail links
US> for each individual echomail area.

sorry... all outside policy and the basic network as design by tom jennings...
someone has fed you a meal of something that contains no nutrients...

US> So what you have to take care off while exporting echomails is to
US> dumb export mails multiple times .. to each linked connection
US> system for a specific echoarea. You don't have to take much care
US> about ftn addressing while exporting echomails .. there are only 2
US> aka's that counts .. the own aka and the link partners aka -
US> individual for each echoarea.

i think i understand what you are trying to say but you are going the long way
about explaining it and making things more complicated then they have to be :?

US> Otherwise in netmail handling ....
US> do you have a default netmail routing ? is the netmail flagged
US> crash or direct? then you have to seperate netmail delivery
US> destination system from the default uplink system (different
US> destination aka's) while exporting a netmail. For the configured
US> netmail uplink there maybe a packet password defined, this doesn't
US> count for the crash netmail delivery to an unconfigured netmail
US> recipient. For a direct delivery of crashmails, you addtl. have to
US> know the direct addressing - phone number to dial, IP address for
US> delivery via IP Then you also have to take care about the status
US> of the destination node. On status Hold or Down, netmail
US> forwarding has to stop and netmail has to be set on hold until the
US> status will change one day to prevent any problems about P407

yes, i'm very aware of all of this... i voted in the process to select if P4.07
would be ratified or not... yes, i'm one of those "old timers" ;)

US> 2.1.7 Not Routing
US> Mail
US> [...]
US> 2.1.7 Not Routing Mail (2nd block)
US> [...]
US> "If you do not forward a message when you previously agreed to
US> perform such routing, the message must be returned to the sysop of
US> the node at which it entered FidoNet with an explanation of why it
US> was not forwarded. (It is not necessary to return messages which
US> are addressed to a node which is not in the current nodelist.)
US> Intentionally stopping an in-transit message without following this
US> procedure constitutes annoying behavior. In the case of a failure
US> to forward traffic due to a technical problem, it does not become
US> annoying unless it persists after being pointed out to the sysop."

US> so the recommendation is to start a nodelist lookup about the
US> destinations aka status before sending out a netmail to fidonet's
US> world ....

yes... yes yes yes... but what does this have to do with packing netmail in a
pkt with echomail? what does it have to do with whether netmail is in a pkt by
itself or if there are multiple netmails in the same pkt? all of the above that
you point to is all handled by the mailer or tosser based on their
configurations... sure, there are numerous details but the software can handle
all of that much faster then any human can...

in any case, though, you do bring up a valid point against binkd and its
non-use of the nodelist as well as its inability to communicate with the system
operator to let them know of a problem delivering mail to an unknown system or
one that is marked as DOWN ;)

[mark lewis]

To: Ulrich Schroeter

On Fri, 04 Oct 2013, Ulrich Schroeter wrote to Mark Lewis:

ml> what is tricky about starting at the topmost VIA line and going

ml> down thru them recording the VIA creation software and the VIA
ml> line creation address? you're only looking for duplicate lines
ml> separated by at least one other VIA line from another system...
ml> nothing difficult about that ;)

US> the built-in macros has only loop, loop2 and loop3 available,

no loops needed in code... one simply starts a list of systems that have seen a
netmail message and indicated such in the VIA lines... if a system is
encountered in that list more than once then there is a loop... that's the most
simplistic explanation... in some cases, a system may inject more than one VIA
line into a message in which case a closer look is needed to be made but even
then the routing loop is easily detected and much easier than what i've seen
shown...

[trim]

US>> as each line requires individual checks (if not the last line)
US>> looping through netmail lines without a definite end marker

ml> hunh? there are several ways to detect the end of the message...

ml> #00 null bytes, EOF or quite simply, the end of the data...
ml> nothing so hard there, either...

US>> is bit tricky with the construction kit that uses macros to
US>> identify kludge lines %MSGID% or %REPLY% extracts the related
US>> msgid or reply kludge lines. and the %LASTVIA% extracts the last
US>> via line .. to identify others then the last via line and LOOP,
US>> LOOP2 or LOOP3 that means "this mail contains one/two/three
US>> itrack vialine of this system in front of others via's"

then this construction kit has problems that need to be worked out and fixed...

ml> wow [laughing] either i'm not understanding or this has been made

ml> more difficult than it needs to be ;)

US> how many parts can have a mail? all specific parts you can select
US> with a keyword selection.

yeah... thanks but no thanks... i think i'll stick with my existing method of
simply reading the message (in my software) and string matching with counters
when necessary for loop detection ;)

[Nicholas Boel]

To: mark lewis
ml> it is quite possible that other systems simply have not seen the problem
ml> and are completely unaware that they may be losing mail...

While this is somewhat of a possibility, these "other systems" used for
testing have been a couple of mine, extensively. I have lost no mail
whatsoever. Although both of my systems also increment on the receiving side
as well.

ml> and it may not be his software but any BAT or shell scripts that he may
ml> have employed...

The point still stands. Any BAT or shell scripts would still be considered
his software. Below for more details on what you probably don't know about..

ml> i don't see where anyone has said that james is doing it wrong... only
ml> that the current method can and apparently has lead to the possible loss
ml> of mail on other systems... in US' case, it was seen in the archiving
ml> portion of his processing... i can see that my processing will also lose
ml> mail when it archives pkts and bundles before processing them... but for
ml> the most part, the archiving for debugging purposes is not critical...
ml> it does, however, show that the problem may appear in other places and
ml> that's not good...

Unfortunately, this started via email before it was even brought into the
echo. I received forwarded emails showing the blame being put on Mystic
itself. Not once was it shown that anything was done to prevent any mail loss
on the system in question. It has /always/ been a Mystic issue since day one,
and one that James should be fixing. I found that to be a little ridiculous,
which is why I argued my point (which you apparantly noticed some of the
exact same questions I was having and referring to, very recently).

[mark lewis]

To: Nicholas Boel

On Sat, 05 Oct 2013, Nicholas Boel wrote to mark lewis:

NB> Unfortunately, this started via email before it was even brought
NB> into the echo.

ahhhh... no wonder it blew into the inferno that it did for a little while...

NB> I received forwarded emails showing the blame being put on Mystic
NB> itself. Not once was it shown that anything was done to prevent
NB> any mail loss on the system in question. It has /always/ been a
NB> Mystic issue since day one, and one that James should be fixing.

yeah, i can just see it now if someone sets up ancient binkleyterm, squish and
maximus and what will happen there with their stoneage packet and bundle
naming... the same thing would happen, actually...

NB> I found that to be a little ridiculous, which is why I argued my
NB> point (which you apparantly noticed some of the exact same
NB> questions I was having and referring to, very recently).

yeah... the main thing i was looking at was that no one can tell anyone what
software they can run or not run... as long as the specs their software states
that it supports are properly supported, then that's it... the complainer can
go pound salt... in the case of the original BSO specs and the filenaming used,
there weren't many folks doing archives of the PKTs and bundles before the toss
simply due to the amount of space needed... it was much harder to track down
problems and determine exactly which piece of software was the culprit or which
ones together caused the problem... one reason why the product ids and similar
software identifiers were created was so that we could look at the pkt or the
messages and know what software had had its fingers in the pie...

my system has two archive directories for each of my several inbound and
outbound directories... i, too, use the simple OS copy or move command to place
the files into the archive directories... my tosser looks to the frontdoor
mailer's inbound directory for inbound mail and i've taken the easy way out in
my hybridization of the system when i added binkd... instead of making another
installation of my tosser for a BSO config and then worrying about if both
tossers were running at the same time, i copy mail from the inbound fileboxes
to the FD inbound... it is quite possible that my system is in the same
prediciment as Uli's... this is something that i have to think deeply about how
to handle the problem... FD and BSO style software do not use the same layout
of inbound or outbound directories and they don't use the same format of
semaphore files so they cannot communicate with each other... if they could,
this would be much easier... i do not plan on dumping FrontDoor because i do
have several daily connections thru it... one of those is the FastEcho author's
system ;) in any case, even if i were to reverse my layout and have BSO be my
main inbound/outbound format, i'd still have to copy files from the FD side of
the fence... and none of that will alleviate the possibility of stoneage
connections dropping off pkts or bundles with the same names... i'm missing
something to make this work and it's 0500 again... i've gone another 24 without
sleep [sigh] oh well, its just another day ;)

[Ulrich Schroeter]

To: Janis Kracht
Hi Janis,

Friday October 04 2013 11:57, you wrote to me:

JK> Hello Uli,

>> ok other question:

>> what does have echomail distribution to do with netmail routing ?

JK> This doesn't have to do with Mystic, but just a note.. over here in
JK> Zone 1, echomail routed netmail often the norm. i.e, if you know
JK> someone's uplink for echomail is 1:261/38, you can route netmail to
JK> that link here. Any of my downlinks can generally get echomail routed
JK> to them, unless they tell me they do not want to pick up any netmail
JK> here. If they tell me to not route netmail to them this way, I can
JK> tell BBBS to not do this.

if no one makes any exceptional netmail routing, the netmail gets delivered the
default netmail route ...
With mostly all link partners I've also short way direct netmail delivery
agreements =:) (with no extra costs ;) )
netmail gets delivered faster then the default routing paths
eg. all netmails to zone 1 receiving my system I deliver directly to 2:261/38
=;) instead of using the default "routing" path via currently still non
existing Zone1 Zonegateway ...

Janis <--- Zone1 Gateway
^ ^
| |
+====== Uli

In POTS days, the Zone1 link produces more costs then the Zonegate link, so the
old-days policy proposes/offers the Zonegateway solution.
Currently running mostly IP links mostly direct calls doesn't produce extra
costs anymore, so we're free to switch to direct delivery instead of old-days
proposed Zonegateway routing

But there are still many sysops who uses the old school routing schema. But
this doesn't mean, that all sysops have to go the old-school way ...

I'm favour of the swarm theory ... that every change within a system will
probably force other intelligent changes in a big system. Little steps may
result in a one days big change ...

But all said, the fact is that netmail is a P2P delivery and echomail is a
one-to-many distribution. This did lead to different deployments for netmail
handling (INTL kludge,MSGID,via kludge) vs. echomail handling (Area
kludge,Origin,MSGID, seenby kludge, path kludge)
In programming style
if type netmail then
handle Netmail Flags Crash, Direct, Hold, Immideate
handle INTL,TOPT,FMPT kludge
handle MSGID, REPLYID kludge
handle via kludge
elseif type echomail then
handle Area kludge
handle MSGID, REPLYID kludge
handle Origin
handle seenby kludge
handle path kludge

except the MSGID / REPLYID kludge all other type specific definitions are
different between netmails and echomails handling.

Don't know what did come first, but makes no difference in this discussion,
to get the netmail vs. echomail philosophy devided. In the days as Frontdoor
was one of the major mailers in the late 80's, early 90's, the netmail bundles
vs. arcmail echomail bundles concept was taken into account by the mailers
(also by Dbridge) to unpack netmails on the fly and trigger a tosser process
for the arcmail bundles of echomails .. to speedup netmail processing probably
good for direct delivery where echomail distribution was configured in the
tossers and netmail routing was configured in the mailer by routing/events
statements that includes the standard routing definitions:

8.6 DEFAULT ROUTING

When no commands in the route file affect a system, implicitly or
explicitly, FD uses a set of built-in rules, these rules are called
default routing.

For mail destined for systems outside the net of the local system,
the default routing is the equivalent of the HOST-ROUTE command.

For mail destined for systems inside the net of the local system,
the default routing is the equivalent of the HUB-ROUTE command.

* For systems with Down status, no default routing applies.

In the days of growing nets, the default netmail routing concept was the
guranteed netmail delivery concept with a minimized hops count.
Looking into current international echo channels default path lines, the count
of hops are not seldom over 10 systems long. Default international echomails
link over the last 10 years from you to me was 4 hop counts ...
Netmail routing I've still switched to direct delivery into Zone 1
long time before, I've also switched some international echomail links with
horrible long paths. And I'm still following the default netmail routing to my
standard uplink if I cannot route directly (for whatever reasons) or don't know
a direct netmail link to the target network, region, zone. In cases where two
links are available for direct delivery I use the most reliable link for direct
delivery. As I've wrote in an earlier mail, I've around 50-60
individual netmail routing links vs. 4 echomail uplinks. All links developed
over the last 10 years.
Whenever I write a netmail, I set the crash flag to get the netmail delivered
as fast as possible, direct as possible to the target system, so I can expect a
reply within probably 24 hours of instead 2-7 days (assuming default hold /
pickup delivery on the others side link end)

If you're looking to the internet services, netmail can be compared to the smtp
email delivery, where echomail can be compared to the nntp delivery mechanism.
A nntp server doesn't has to serve as smtp gateway for the users subscribing to
the nntp server ...

For fidonet systems as endnodes with one uplink only, you can simplify netmail
and echomail forwarding to one uplink only, but than you shouldn't makes you
wonder, that netmail delivery is as slow as echomail delivery =:)

JK> Take care,
JK> Janis
JK> -$- BBBS/Li6 v4.10 Dada-1
JK> $ Origin: Prism bbs (1:261/38)