Is it possible do defer big messages??

[Federico Albesano]

Hi,

We have a little 64K digital link between two sendmail hosts. When a message
with big attachement crosses that link, all other traffic is unacceptably
delayed.
We would like to queue big messages and delivery them in nigth ours.
Probably I need a mean to queue messages in two separate directoryes based
on their size and then run the queue with big messages once at, for example,
midnight, but I don't know how to make the two queue working.
Is this possible using sendmail V8.8?
Have I to write a checkcompat() procedure to do that?
Is there another hack to solve my problem?

Thanks all for help!!

Federico Albesano

[Andrzej Filip]

Federico Albesano wrote:

AFAIK sendmail has no way to differentiate deliveries based on
message size.

The only way I can think of is to use $i (the queue identifier)
as argument for your program ("program" database) which will test
the message size.

The Bat Book 2nd ed. [ http://www.sendmail.org/books.html ]
section 31.10.19 $i variable
section 33.8.14 program database

--
Andrzej (Andrew) A. Filip - Warsaw, Poland
http://bigfoot.com/~anfi | fax2mail: +1(801)327-6278
Postings: http://deja.com/profile.xp?author=Andrzej%20Filip
Private: an...@bigfoot.com | an...@polbox.com | an...@box43.gnet.pl
Business: an...@impaq.com.pl -- IMPAQ Technology Center

[rich...@eskom.co.za]

Andrzej Filip wrote:
>
> Federico Albesano wrote:
>
> > Hi,
> >
> > We have a little 64K digital link between two sendmail hosts. When a message
> > with big attachement crosses that link, all other traffic is unacceptably
> > delayed.
> > We would like to queue big messages and delivery them in nigth ours.
> > Probably I need a mean to queue messages in two separate directoryes based
> > on their size and then run the queue with big messages once at, for example,
> > midnight, but I don't know how to make the two queue working.
> > Is this possible using sendmail V8.8?
> > Have I to write a checkcompat() procedure to do that?
> > Is there another hack to solve my problem?
> >
> > Thanks all for help!!
> >
> > Federico Albesano
>
> AFAIK sendmail has no way to differentiate deliveries based on
> message size.

Correct, but a small little patch to the sources of 8.9.3 can do this
for you. Basically what this does is to check the message size and queue
the message if it is larger that a certain size. Here is the patch.
Apply as patch -p0 in the sendmail-8.9.3 directory. Change the 1024*1024
(or 1meg) to whatever size you want to queue. Messages larger than this
size will be queued, and not immediately delivered. You can then grab
them out of the queue at regular intervals:

--- src/deliver.c Mon Sep 13 11:01:09 1999
+++ src/deliver.c.orig Mon Sep 13 11:31:51 1999
@@ -99,8 +99,6 @@
if (mode != SM_VERIFY && mode != SM_DEFER &&
shouldqueue(e->e_msgpriority, e->e_ctime))
mode = SM_QUEUE;
- if (e->e_msgsize > 1024*1024)
- mode = SM_QUEUE;
}

if (tTd(13, 1))

Next, assuming your queue directory is /var/spool/mqueue, create another
directory called /var/spool/slowmqueue. Next, create a script called
/usr/local/sbin/mailsize and place this script in there:

#!/bin/sh

# Move mail messages over a certain size to another spool directory
# for later delivery.

# The size here must match the size compiled into the patched sendmail

BASE_QUEUE=/var/spool
SPOOL=mqueue
SLOWSPOOL=slowmqueue

find $BASE_QUEUE/$SPOOL/df* -size +1024k -printf %f\\n | cut -b2- >
/tmp/queue

while read FILE
do

# check if there is a xfile, if not move file
if [ -r $BASE_QUEUE/$SPOOL/d$FILE -a ! -f $BASE_QUEUE/$SPOOL/x$FILE ]
then
mv $BASE_QUEUE/$SPOOL/?$FILE $BASE_QUEUE/$SLOWSPOOL
fi

done < /tmp/queue

Basically what the script does is to find all messages > 1meg in
/var/spool/mqueue that do not have associated xf* files (ie, the message
is busy being processed) and moves them to /var/spool/slowmqueue for
later delivery.

Next, create two cron entries:

# Run slowmqueue every 30 mins afterhours
0,30 19,20,21,22,23,0,1,2,3,4,5 * * * /usr/sbin/sendmail
-oQ/var/spool/slowmqueue -q > /dev/null 2>/dev/null
# Move large messages to slow queue every 5 minutes during the day
0,5,10,15,20,25,30,35,40,45,50,55 7-18 * * * /usr/local/sbin/mailsize >
/dev/null 2>/dev/null

The first entry runs through the slowmqueue every 30 minutes afterhours.
The second one moves the messages from the normal mqueue to the
slowmqueue every 5 minutes during the day. You can change the times as
desired.

Note: This was written on Linux Redhat. The find in the mailsize script
will probably not work on other Unix'es and may need to be recoded.
Also, as a side effect, incomming mail will also be queued! You can
probably modify the if statement in the c source to exclude your own
domain. Well, this little trick works for me - during the average day I
queue around 3gigs of mail in the slowmqueue directory for afterhours
delivery! Nobody has complained yet!

Can this be added to the sendmail site as a contrib maybe? I have seen
this come up a couple of times - it may even be a FAQ.

[Howard Soper]

Sorry to be off topic a bit - but is it possible to queue outgoing ( aka ISP
bound mail ) but not local messages.

I have everything deferred for hourly delivery but it would be nice if local
mail was NOT defered.

Cheers
Howard Soper

Federico Albesano <fede...@proton.it> wrote in message
news:7uhej1$s5g$1...@fe1.cs.interbusiness.it...

[Andrzej Filip]

rich...@eskom.co.za wrote:

> [...]

> Correct, but a small little patch to the sources of 8.9.3 can do this
> for you. Basically what this does is to check the message size and queue
> the message if it is larger that a certain size. Here is the patch.
> Apply as patch -p0 in the sendmail-8.9.3 directory. Change the 1024*1024
> (or 1meg) to whatever size you want to queue. Messages larger than this
> size will be queued, and not immediately delivered. You can then grab
> them out of the queue at regular intervals:
>
> --- src/deliver.c Mon Sep 13 11:01:09 1999
> +++ src/deliver.c.orig Mon Sep 13 11:31:51 1999
> @@ -99,8 +99,6 @@
> if (mode != SM_VERIFY && mode != SM_DEFER &&
> shouldqueue(e->e_msgpriority, e->e_ctime))
> mode = SM_QUEUE;
> - if (e->e_msgsize > 1024*1024)
> - mode = SM_QUEUE;
> }
>
> if (tTd(13, 1))

> [...]

1) Thanks for the simple but efficient patch

2) It seems that in some situations your patch may change delivery
mode of big messages from "deferred" to "queue" causing unnecessary
DNS queries

[rich...@eskom.co.za]

Andrzej Filip wrote:
>
> rich...@eskom.co.za wrote:
>
> > [...]
> > Correct, but a small little patch to the sources of 8.9.3 can do this
> > for you. Basically what this does is to check the message size and queue
> > the message if it is larger that a certain size. Here is the patch.
> > Apply as patch -p0 in the sendmail-8.9.3 directory. Change the 1024*1024
> > (or 1meg) to whatever size you want to queue. Messages larger than this
> > size will be queued, and not immediately delivered. You can then grab
> > them out of the queue at regular intervals:
> >
> > --- src/deliver.c Mon Sep 13 11:01:09 1999
> > +++ src/deliver.c.orig Mon Sep 13 11:31:51 1999
> > @@ -99,8 +99,6 @@
> > if (mode != SM_VERIFY && mode != SM_DEFER &&
> > shouldqueue(e->e_msgpriority, e->e_ctime))
> > mode = SM_QUEUE;
> > - if (e->e_msgsize > 1024*1024)
> > - mode = SM_QUEUE;
> > }
> >
> > if (tTd(13, 1))
> > [...]
>
> 1) Thanks for the simple but efficient patch

Pleasure! Nobody could help me the first time I asked this same
question, so I went and hacked around the sources myself. The only
problem that I have is that according to the manual, the shouldqueue()
function should control if messages are queued or not. Well, the message
environment does not appear to be available to the shouldqueue()
function. Anybody with some contact with the developers to explain this
anomaly would be appreciated.

> 2) It seems that in some situations your patch may change delivery
> mode of big messages from "deferred" to "queue" causing unnecessary
> DNS queries

I am not quite sure what you mean, but this little change should
probably "fix" it:

--- src/deliver.c.test Wed Oct 20 17:50:56 1999

+++ src/deliver.c.orig Mon Sep 13 11:31:51 1999

@@ -99,9 +99,6 @@

if (mode != SM_VERIFY && mode != SM_DEFER &&
shouldqueue(e->e_msgpriority, e->e_ctime))
mode = SM_QUEUE;

- if (mode != SM_VERIFY && mode != SM_DEFER &&
- e->e_msgsize > 1024*1024)

- mode = SM_QUEUE;
}

if (tTd(13, 1))

Now only if we can make the size a parameter in the config file instead
of being statically configured!