MikroTik Setup - Queues & QoS

[Jacques Rossouw]

Hi again!

Next step is to incorporate a Queuing system in addition to the routing marks.  I'm running into some trouble with overlapping marks, and could use some wizdom...

Just a reminder, my setup has a single LAN area for my home network, and two outgoing interfaces set up to two different ISPs.  The idea so far has been to split certain types of traffic by origin / destination / ports to each of the two interfaces (downloads go over the uncapped line, http and gaming goes over the fast capped line)

1. I'm currently marking each connection coming in over an interface with that interface's connection mark, and have some rules to mark outgoing connections for either interface, as described.

2. I then use a routing-mark based on the connection mark for use in the routing table.

3. All of this happens in prerouting mangle, and it's working 100%

Now I want to add packet marks in order to split traffic by type into the queueing system, but most examples also suggest using connection marks in these rules, and then marking the packets based on the connection marks before it goes to the queue.

This also has to happen in prerouting.

Obviously, I have a conflict here.  Each connection can only carry a single connection mark.

So I've rewritten the rules to skip the routing connection marks, and going straight to the routing marks.  The idea is that once a connection is established over an interface with its own public IP address, all further packets will follow that same path?  But I'm missing something, and it's not working.

The subsequent plan was to use connection marks and packet marks for my queue system after that.

Any advice?  Any idea where I'm going wrong?  Is it possible to do away with connection marks and still route over my two interfaces?  Or should I resign myself to having to traverse a whole list of 20 packet marking rules for the queue each time for EACH packet?

A bit stumped...

J

--
Vincit Omnia Veritas

[Jacques Rossouw]

Back!

Ok, I've solved this.  It's a bit more complicated that I'd have liked, but it's working sweetly without taxing the router too much.  Basically, I'm first assigning a routing mark to unmarked packets based on the gateway I want to use, then I assign a packet mark based on the QoS I want for that connection type.  Lastly, I assign a connection mark to the entire connection based on these two marks, and all further packets in that connection get the same routing/packet mark without having to revisit all my rules.

I'm attaching my Mangle rule-set for anyone wanting to do something similar, as well as a screen grab of my queue config.  If I've left anything out, hollar.

I picked up a few things that I'd like some help with:

1. Routing Marks.  I'm assigning routing marks to packets based on destination IP, but I can't turn off "passthrough" since I'm assigning other marks (routing and connection) later on in the mangle table.  I've written rules in the form "if packet doesn't have a routing mark yet, and it's destined for IP xxx.xxx.xxx.xxx, give it a mark M".  

The problem is that it seems that I can't test for a blank routing mark.  Packet marks have a default value (no-mark), but while I assumed the same is true for routing marks (main), it does not appear that tests for that are successful.  I've solved the issue by marking all packets that enter the table with a default value before I start processing, but it seems messy.  Am I missing something?

2. Static DNS. I'm maintaining a list of static DNS entries matched to public hostnames and firewall address-lists, for use in my firewall rules.  i.e. Each such hostname has both a fw-list entry and a static DNS entry in my router.  The fw-list entry is referenced in the firewall rules, while I keep the static DNS entry for hostnames that map to any of a range of IPs at resolution time.  This ensures that clients behind my firewall get the same IP that I have a firewall rule written for, when they contact that host.

My problem is that when I want to check if these hostnames need an IP update, the router would also use the static DNS entry when resolving, instead of querying the public DNS.  This is a small problem that I'm avoiding by disabling the static DNS rule before resolving, but this fills up my logs with noise ("user jacques changed a static DNS rule") every 5 minutes.  I can't find proper documentation for the :resolve command, but it seems that you should be able to specify the DNS to use via the "server" directive, similar to the linux command "dig @8.8.8.8 www.google.com".  But it's not working.  Can anyone help with the correct syntax, or is the disabling method my only solution?  To clarify, I'd like to resolve a hostname externally, while ignoring static and/or cached DNS entries on the local router.

3. Failover design.  I've created routing tables based on my routing marks.  The simplified tables: 

MarkA

10.1.1.0/24 bridge-local distance=1

0.0.0.0/0 pppoe-wafr distance=1

0.0.0.0/0 pppoe-mweb distance=10

MarkB

10.1.1.0/24 bridge-local distance=1

0.0.0.0/0 pppoe-mweb distance=1

0.0.0.0/0 pppoe-wafr distance=10

All pppoe routes have ping checking enabled.  I imagine that the lowest cost default route will ALWAYS be used, unless a route fails a ping test and becomes inactive.  If the low cost route becomes inactive, the higher cost route will be used.

I've looked at some designs online, where redundant external hosts (like www.google.com) are pinged THROUGH each interface, but I don't know why that would be needed.  Is this design sufficient?  What are the downsides to this approach and/or how are you guys doing it?

Thanks!

J

[Tim Haak]

Hi

1. You should be using something else to set routing marks. Ie connection tracking marks. So that if you route something down one path it wills stay going down that path. Also you should only be setting connection marks on first packet only unless your doing something specifically complicated (we once were changing it to alter the shaping once we had clasified what was being downloaded).

Having a catch all to set unkown marks to something is also a good practice as 1 you find out what you haven't thought about. 2 it becomes you unsafe traffic eventually.

3. The reason why people ping a system far away ie google. Is if something goes wrong upstream with the isp it will then switch. Ie they loose overseas connectivity. Google isn't your best for this anymore as there are now local caches.

Whether its worth it or not depends on how much you care

[Jacques Rossouw]

Thanks, Tim.  Makes sense.

In my case, I have to use two rounds of marking to determine the correct connection mark.  First determining factor is based on where the traffic is going, and the second factor is what type (quality) of service is required.  During the first section, I can't yet assign a connection mark, since I use the fact that the connection is as-yet unmarked during the second marking phase.  I do assign the connection mark during the second phase, and no subsequent packets needs to go through this treatment.

And I actually do have a rule to de-prioritize a previous mark to a lower quality! ;-)  I normally give HTTP traffic a high priority, but if a single connection exceeds a determined threshold I drop the connection priority a few levels.  This catches youtube and other HTTP downloads.

After the weekend I feel like I'm pretty much set for now, almost time to return to my 3d Printer design after this little detour. ;-)

I've written a few scripts to expand the base functionality, which I would like to make available for everyone to use.  Please feel free to use as desired, and would love to see improvements from others that do.  Here is a list of scripts I'm attaching:

• ad-ports.rsc - Script to list all IP addresses directly connected to the ports on the router, as well as which ports they are connected to.
• ad-sniff.rsc - Script to find current bandwidth usage of local clients on the network. Useful to find sudden peak users impacting service on the network.
• blacklist_report.rsc - Checks the list of blacklisted IPs in the firewall, and if a new entry was made in the last [scheduleFreq] minutes, email the administrator with the contents of the blacklist, and a dump of the firewall logs.
• bw-wafr.rsc - Logs and accumulates the usage counters from the queue to local storage for tracking, backup and archiving.  Stores basic statistics relative to bandwidth quotas.  The script also controls FW Mangle rules when quota is exceeded, to prevent exceeding quota.
• dns-update.rsc - Maintains static DNS entries for DHCP local hosts, based on client MAC addresses. Static hosts are maintained by a list of global environment variables, starting with "shost" followed by the client MAC address. This feature allows local clients to refer to other local hosts by their hostname, instead of by their IP address.
• fwip-update - Script renews firewall address-list IPs based on hostname in their comments. This allows the use of variables in firewall rules that update as external IPs change.  The script also maintains a matching list of static DNS entries for these hostnames to ensure that clients resolve the same IP as controlled for in the firewall, when doing a DNS lookup for these hosts.
• ntp-update.rsc - Maintains an updated NTP time server IP for the built-in NTP agent.  This script was written with NTP pools in mind, where the published NTP IP changes on a regular basis.
• dns-noip.rsc - Script to maintain a dynamic IP using a public DNS provider.  This script also maintains a firewall address-list with an expiration time, the ensure that the DNS is refreshed at a minimum interval even if the ISP assigned IP haven't changed in that time.
• dns-dnsexit.rsc - Script to maintain a dynamic IP using a public DNS provider.  This script also maintains a firewall address-list with an expiration time, the ensure that the DNS is refreshed at a minimum interval even if the ISP assigned IP haven't changed in that time.

Hope someone finds it useful!

Regards,

J