ap/app/iproute2/iproute2-3.4.0/doc/actions/mirred-usage

Very funky action. I do plan to add to a few more things to it
This is the basic stuff. Idea borrowed from the way ethernet switches
mirror and redirect packets. The main difference with say a vannila
ethernet switch is that you can use u32 classifier to select a
flow to be mirrored. High end switches typically can select based
on more than just a port (eg a 5 tuple classifier). They may also be
capable of redirecting.

Usage: 

mirred <DIRECTION> <ACTION> [index INDEX] <dev DEVICENAME> 
where: 
DIRECTION := <ingress | egress>
ACTION := <mirror | redirect>
INDEX is the specific policy instance id
DEVICENAME is the devicename

Direction:
- Ingress is not supported at the moment. It will be in the
future as well as mirror/redirecting to a socket. 

Action:
- Mirror takes a copy of the packet and sends it to specified
dev ("port" in ethernet switch/bridging terminology)
- redirect
steals the packet and redirects to specified destination dev.

What NOT to do if you dont want your machine to crash:
------------------------------------------------------

Do not create loops! 
Loops are not hard to create in the egress qdiscs.

Here are simple rules to follow if you dont want to get
hurt:
A) Do not have the same packet go to same netdevice twice
in a single graph of policies. Your machine will just hang!
This is design intent _not a bug_ to teach you some lessons. 

In the future if there are easy ways to do this in the kernel
without affecting other packets not interested in this feature
I will add them. At the moment that is not clear.

Some examples of bad things NOT to do:
1) redirecting eth0 to eth0
2) eth0->eth1-> eth0
3) eth0->lo-> eth1-> eth0

B) Do not redirect from one IFB device to another.
Remember that IFB is a very specialized case of packet redirecting
device. Instead of redirecting it puts packets at the exact spot
on the stack it found them from.
Redirecting from ifbX->ifbY will actually not crash your machine but your 
packets will all be dropped (this is much simpler to detect
and resolve and is only affecting users of ifb as opposed to the
whole stack).

In the case of A) the problem has to do with a recursive contention
for the devices queue lock and in the second case for the transmit lock.

Some examples:
-------------

1) Mirror all packets arriving on eth0 to be sent out on eth1.
You may have a sniffer or some accounting box hooked up on eth1.
 
---
tc qdisc add dev eth0 ingress
tc filter add dev eth0 parent ffff: protocol ip prio 10 u32 \
match u32 0 0 flowid 1:2 action mirred egress mirror dev eth1
---

If you replace "mirror" with "redirect" then not a copy but rather
the original packet is sent to eth1.

2) Host A is hooked  up to us on eth0

# redirect all packets arriving on ingress of lo to eth0
---
tc qdisc add dev lo ingress
tc filter add dev lo parent ffff: protocol ip prio 10 u32 \
match u32 0 0 flowid 1:2 action mirred egress redirect dev eth0
---

On host A start a tcpdump on interface connecting to us.

on our host ping -c 2 127.0.0.1

Ping would fail since all packets are heading out eth0
tcpudmp on host A would show them

if you substitute the redirect with mirror above as in:
tc filter add dev lo parent ffff: protocol ip prio 10 u32 \
match u32 0 0 flowid 1:2 action mirred egress mirror dev eth0

Then you should see the packets on both host A and the local
stack (i.e ping would work).

3) Even more funky example:

#
#allow 1 out 10 packets on ingress of lo to randomly make it to the 
# host A (Randomness uses the netrand generator)
#
---
tc filter add dev lo parent ffff: protocol ip prio 10 u32 \
match u32 0 0 flowid 1:2 \
action drop random determ ok 10\
action mirred egress mirror dev eth0
---

4)
# for packets from 10.0.0.9 going out on eth0 (could be local 
# IP or something # we are forwarding) - 
# if exceeding a 100Kbps rate, then redirect to eth1 
#

---
tc qdisc add dev eth0 handle 1:0 root prio
tc filter add dev eth0 parent 1:0 protocol ip prio 6 u32 \
match ip src 10.0.0.9/32 flowid 1:16 \
action police rate 100kbit burst 90k ok \
action mirred egress mirror dev eth1
---

A more interesting example is when you mirror flows to a dummy device
so you could tcpdump them (dummy by defaults drops all packets it sees).
This is a very useful debug feature.

Lets say you are policing packets from alias 192.168.200.200/32
you dont want those to exceed 100kbps going out.

---
tc qdisc add dev eth0 handle 1:0 root prio
tc filter add dev eth0 parent 1: protocol ip prio 10 u32 \
match ip src 192.168.200.200/32 flowid 1:2 \
action police rate 100kbit burst 90k drop
---

If you run tcpdump on eth0 you will see all packets going out
with src 192.168.200.200/32 dropped or not (since tcpdump shows
all packets being egressed).
Extend the rule a little to see only the packets making it out.

---
tc qdisc add dev eth0 handle 1:0 root prio
tc filter add dev eth0 parent 1: protocol ip prio 10 u32 \
match ip src 192.168.200.200/32 flowid 1:2 \
action police rate 10kbit burst 90k drop \
action mirred egress mirror dev dummy0
---

Now fire tcpdump on dummy0 to see only those packets ..
tcpdump -n -i dummy0 -x -e -t

Essentially a good debugging/logging interface (sort of like
BSDs speacialized log device does without needing one).

If you replace mirror with redirect, those packets will be
blackholed and will never make it out. 

cheers,
jamal