Mailserv5.3 devel

[David Gnojek]

Hi all,

testing installation available in devel-5.3 branch

#git clone git://github.com/mailserv/mailserv.git
#cd /var/mailserv
#./install/install.sh --devel devel-5.3

Cheers

David

[Wylie Bayes]

Any problems with this?

Everything works on 5.3 ?

I was just getting ready to email the mailserv googlegroups email asking if it has been updated for 5.2 / 5.3 yet.

If so i'll get this sucker going again.   Had it running on 5.1 fully.

-Wylie

[Kim Thostrup]

Hi Wylie,

Did you manage to verify/test the install on 5.3 ?

Thanks

/Kim

-- 

Kim Thostrup

Sent with Sparrow

--
You received this message because you are subscribed to the Google Groups "mailserv" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mailserv+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Marcus Bointon]

I just took a look at the 5.3 branch compared to my own devel branch (in which I was working on 5.3) and there are a lot of differences. I've done a big rebase from the master 5.3 branch to tidy it up a bit. I'm testing the install now.

My branch is here: https://github.com/Synchro/mailserv/tree/devel

I think there are still some things pointing at the built-in nginx, which we tried to use for a bit but abandoned in favour of the add-on package.

Marcus

[Kim Thostrup]

With that in mind i guess the recommended version to go for with a upgrade from 5.1 must be 5.2. Do you agree ?

Thanks

/Kim

[Marcus Bointon]

On 27 Sep 2013, at 16:16, Kim Thostrup <kimth...@gmail.com> wrote:

With that in mind i guess the recommended version to go for with a upgrade from 5.1 must be 5.2. Do you agree ?

I'm about to do a migration from 4.8 to 5.3 as there's lots of nice stuff in it. I'm not sure of the state of 5.2 - it took a long time to come out and by the time it was marked as stable, OBSD 5.3 was already out. I know that most of the work I've done was for 5.3, and 5.4 is just around the corner...

Marcus

[David Gnojek]

v5.3 works me fine. so have set a stable branch 5.3 for it.

About v5.2: There had been couple of troubles to overcome but it's working now, but still stay in devel branch. I focused rather on v5.3 because it was more actual that time.

basically upgrade from 5.1. to 5.3 should works too.

Cheers

David

--

[Marcus Bointon]

On 30 Sep 2013, at 14:35, David Gnojek <gno...@gmail.com> wrote:

v5.3 works me fine. so have set a stable branch 5.3 for it. 

About v5.2: There had been couple of troubles to overcome but it's working now, but still stay in devel branch. I focused rather on v5.3 because it was more actual that time. 

basically upgrade from 5.1. to 5.3 should works too.

I've been doing some more work on my 5.3 fork, ironing out various glitches. It now has these features over the stock 5.3:

Improved PHP setup for roundcube

Adds gd and imagemagick, enables various image handling features in roundcube.

Uses mysqli extension with mysqlnd driver, connected via socket  (safer, faster, less memory)

Uses memcache for sessions

Shows more logs in admin

Shows more status indicators in admin (memcached, nginx, php)

Postfix transport config to throttle deliveries to all yahoo domains

It could do with more testing, so please have a play: https://github.com/Synchro/mailserv/tree/devel

Marcus

[Kim Thostrup]

Hi Marcus,

It sounds really tempting, and nice features you worked in there.

Are you running your build in production ?

/Kim

--

[Marcus Bointon]

On 1 Oct 2013, at 20:02, Kim Thostrup <kimth...@gmail.com> wrote:

It sounds really tempting, and nice features you worked in there.

Are you running your build in production ?

Well, yes and no! It all works beautifully locally in VMWare fusion, but I just ran it on my new KVM production instance and starting MySQL failed, which broke quite a few things. Fortunately these are about the last things to be done and I'm migrating my databases from an old 4.8 instance anyway, and it worked fine when I started it manually. I've saved the logs from where it failed, but that area is not one I've changed much, so I don't know the cause.

Something I've noticed isn't working well with the stock setup is that the spamcontrol database doesn't get cleared out much - my bayes_token table has 6 million records and is very slow to do anything. I think the pruning needs to be more aggressive there. My 4.8 setup isn't very happy - I've gradually had to disable various things as they were too unreliable, so I'm hoping that the 5.3 setup will be better. The new roundcube is certainly very pretty!

Marcus

[Kim Thostrup]

Thanks for your feedback Marcus, it's appreciated :)

/Kim

--

[Marcus Bointon]

More updates on 5.3.

I found more issues with my migration from 4.8.

Postfix changed how it configures relay permissions in 2.10, and I found that it was missing in the main.cf file so it would not allow relaying by authenticated users (giving "5.7.1 Relay access denied" errors). The postfix release notes make this clear:
http://de.postfix.org/ftpmirror/official/postfix-2.10.0.RELEASE_NOTES under "Major changes - relay safety". I've fixed the config in my github fork. I also removed the setting of 'transport_destination_recipient_limit' which was causing this error:

/usr/local/sbin/postconf: warning: /etc/postfix/main.cf: unused parameter: transport_destination_recipient_limit=1

This directive is meaningless as it's generic, intended to be renamed for specific transports like dovecot_destination_recipient_limit.

After changing those it's now working for inbound and outbound. Commit is here: https://github.com/Synchro/mailserv/commit/ef8af8acd7a86ee079a92eb10d9c26dd2fb66b28

Throughout this migration, the bayes_token table in my original spamcontrol database was a real problem. It had grown to 400M and 6 million records - clearly sa-learn was not pruning it. It was causing major slowdowns because it was a MyISAM table - it gets a lot of writes and locks the table every time, leading to serialisation of all spamassassin processes, major backlogs and terrible iowait. It's really important that this is changed to use InnoDB as it will remove the locking issue. In my own config I've changed all my tables to InnoDB. In the default config it's hard to prune the database manually because the fields you'd use to do so are not indexed usefully, and a table scan of a big table like that can take literally days on a small server, hanging up everything.

One thing I'm left with is that if I click 'admin' in roundcube for a designated domain admin, I get "We're sorry, but something went wrong.", but I'm not seeing anything logged relating to that. Any ideas?

Marcus

[electricart]

Hi Marcus,

I recently installed your branch. I installed it a few times (had to reinstall the server a few times for a few different reasons). The first time I had it installed, Roundcube worked fine, however the most recent time I installed it, Roundcube just ends up a blank page, and I get the following error in /var/log/messages;

roundcube: DB Error: Failed to connect to memcached. Please check configuration in /var/www/webmail/roundcubemail-0.9.5/program/lib/Roundcube/rcube_session.php on line 77 (GET /)

Any idea how I can fix this? Would running the install command again cause problems?

I believe the command I used to install was;

git clone https://github.com/Synchro/mailserv
/var/mailserv/install/install.sh --devel 5.3

It appeared to install without a problem.

Any help on this would be greatly appreciated!

kind regards,
Tristan.

[electricart]

Arg. Now it appears I've really ruined things. I tried running an update using;

./scripts/update

and stopped when it was trying to re-install postfix for some reason. Now postfix doesn't run, and trying to run the update again only updates the ClamAV definitions. Trying to run;

/var/mailserv/install/install.sh --devel 5.3

Now for some reason results in this;

# /var/mailserv/install/install.sh --devel 5.3
set -e
+ set -e
if [[ `uname -s` != "OpenBSD" ]]; then
  echo "This only works on OpenBSD!"
  exit 1
fi
+ uname -s

export MAILSERV_DEVEL=1
+ export MAILSERV_DEVEL=1

# git checkout branch for supported OpenBSD version or development branch
# can also specify '--devel <branchname>' to use a specific named branch
if [ "`echo $1`" == "--devel" ]; then
  if [ "`echo $2`" != "" ]; then
    checkout_switch=$2
  else
    checkout_switch="devel"
  fi
  # detect changes in devel branch
  branch_changes=`git --git-dir=/var/mailserv/.git --work-tree=/var/mailserv status -s`

  # Set an env var so scripts can know we're in devel mode
  export MAILSERV_DEVEL=1

elif [ "`echo $1`" == "--help" ]; then
  echo "Usage: install.sh [OPTION]"
  echo " "
  echo "    Without option it installs a stable Mailserv if compatible with your OBSD version."
  echo "    --devel    use only in case you want to try to intall on unsupported OBSD version"
  echo "  or "
  echo "    --devel <another_devel_branch_name> "
  echo "    --help     show this info"
  echo " "
  exit 1
else
  checkout_switch=`uname -r`
fi
+ echo --devel
+ [ --devel == --devel ]
+ echo 5.3
+ [ 5.3 !=  ]
+ checkout_switch=5.3
+ git --git-dir=/var/mailserv/.git --work-tree=/var/mailserv status -s
+ branch_changes=?? ../account/public/blank.html
?? ../awstats/awstats012014.fuzzygum.electricart.com.au.txt
?? ../git.core
+ export MAILSERV_DEVEL=1

if [ ! -z "$branch_changes" ]; then
  echo " "
  echo "Code has been changed manually in Devel branch, system will not replace it from git hub."
  echo "Please add file(s) and commit changes"
  echo "    cd /var/mailserv "
  echo "    git add <filename>"
  echo "    git commit -a"
  echo " "
  echo "This file(s) has been changed or added:"
  echo " $branch_changes "
  echo " "
  exit 1
else
  # switch to branch 
  git --git-dir=/var/mailserv/.git --work-tree=/var/mailserv checkout $checkout_switch 2>/dev/null                

  if [[ `echo $?` -ne 0 ]]; then                             # and if that fails
   echo "Mailserv is not yet supported on OpenBSD `uname -r`, please use a supported version of OpenBSD or --devel parameter"
   exit 1
  fi
fi
+ [ ! -z ?? ../account/public/blank.html
?? ../awstats/awstats012014.fuzzygum.electricart.com.au.txt
?? ../git.core ]
+ echo 
 
+ echo Code has been changed manually in Devel branch, system will not replace it from git hub.
Code has been changed manually in Devel branch, system will not replace it from git hub.
+ echo Please add file(s) and commit changes
Please add file(s) and commit changes
+ echo     cd /var/mailserv
    cd /var/mailserv
+ echo     git add <filename>
    git add <filename>
+ echo     git commit -a
    git commit -a
+ echo 
 
+ echo This file(s) has been changed or added:
This file(s) has been changed or added:
+ echo  ?? ../account/public/blank.html
?? ../awstats/awstats012014.fuzzygum.electricart.com.au.txt
?? ../git.core
 ?? ../account/public/blank.html
?? ../awstats/awstats012014.fuzzygum.electricart.com.au.txt
?? ../git.core
+ echo 
 
+ exit 1
root@fuzzygum:/var/mailserv/install # /var/mailserv/install/sh install.sh --devel 5.3
ksh: /var/mailserv/install/sh: not found
root@fuzzygum:/var/mailserv/install # sh /var/mailserv/install/install.sh --devel 5.3
set -e
+ set -e
if [[ `uname -s` != "OpenBSD" ]]; then
  echo "This only works on OpenBSD!"
  exit 1
fi
+ uname -s

export MAILSERV_DEVEL=1
+ export MAILSERV_DEVEL=1

# git checkout branch for supported OpenBSD version or development branch
# can also specify '--devel <branchname>' to use a specific named branch
if [ "`echo $1`" == "--devel" ]; then
  if [ "`echo $2`" != "" ]; then
    checkout_switch=$2
  else
    checkout_switch="devel"
  fi
  # detect changes in devel branch
  branch_changes=`git --git-dir=/var/mailserv/.git --work-tree=/var/mailserv status -s`

  # Set an env var so scripts can know we're in devel mode
  export MAILSERV_DEVEL=1

elif [ "`echo $1`" == "--help" ]; then
  echo "Usage: install.sh [OPTION]"
  echo " "
  echo "    Without option it installs a stable Mailserv if compatible with your OBSD version."
  echo "    --devel    use only in case you want to try to intall on unsupported OBSD version"
  echo "  or "
  echo "    --devel <another_devel_branch_name> "
  echo "    --help     show this info"
  echo " "
  exit 1
else
  checkout_switch=`uname -r`
fi
+ echo --devel
+ [ --devel == --devel ]
+ echo 5.3
+ [ 5.3 !=  ]
+ checkout_switch=5.3
+ git --git-dir=/var/mailserv/.git --work-tree=/var/mailserv status -s
+ branch_changes=?? ../account/public/blank.html
?? ../awstats/awstats012014.fuzzygum.electricart.com.au.txt
?? ../git.core
+ export MAILSERV_DEVEL=1

if [ ! -z "$branch_changes" ]; then
  echo " "
  echo "Code has been changed manually in Devel branch, system will not replace it from git hub."
  echo "Please add file(s) and commit changes"
  echo "    cd /var/mailserv "
  echo "    git add <filename>"
  echo "    git commit -a"
  echo " "
  echo "This file(s) has been changed or added:"
  echo " $branch_changes "
  echo " "
  exit 1
else
  # switch to branch 
  git --git-dir=/var/mailserv/.git --work-tree=/var/mailserv checkout $checkout_switch 2>/dev/null                

  if [[ `echo $?` -ne 0 ]]; then                             # and if that fails
   echo "Mailserv is not yet supported on OpenBSD `uname -r`, please use a supported version of OpenBSD or --devel parameter"
   exit 1
  fi
fi
+ [ ! -z ?? ../account/public/blank.html
?? ../awstats/awstats012014.fuzzygum.electricart.com.au.txt
?? ../git.core ]
+ echo 
 
+ echo Code has been changed manually in Devel branch, system will not replace it from git hub.
Code has been changed manually in Devel branch, system will not replace it from git hub.
+ echo Please add file(s) and commit changes
Please add file(s) and commit changes
+ echo     cd /var/mailserv
    cd /var/mailserv
+ echo     git add <filename>
    git add <filename>
+ echo     git commit -a
    git commit -a
+ echo 
 
+ echo This file(s) has been changed or added:
This file(s) has been changed or added:
+ echo  ?? ../account/public/blank.html
?? ../awstats/awstats012014.fuzzygum.electricart.com.au.txt
?? ../git.core
 ?? ../account/public/blank.html
?? ../awstats/awstats012014.fuzzygum.electricart.com.au.txt
?? ../git.core
+ echo 
 
+ exit 1


Any help with this would be very much appreciated. I'm really stuck now!

kind regards,
Tristan.

[electricart]

I manually reinstalled postfix and that seems to have worked, but I'm still at a loss of how to fix Roundcube or how to update Mailserv (when it may become necessary)...?

[Marcus Bointon]

On 7 Jan 2014, at 03:18, electricart <em...@electricart.com.au> wrote:

I manually reinstalled postfix and that seems to have worked, but I'm still at a loss of how to fix Roundcube or how to update Mailserv (when it may become necessary)...?

The installer script is not set up to do upgrades. Generally you're better off doing a clean install and migrating mailboxes. You can update standard installed packages with pkg_add, but there isn't an easy system for non-system packages like roundcube.

Marcus