set_ts_lo.sh Traffic Shaping

80 views
Skip to first unread message

Geoff

unread,
Dec 8, 2011, 11:07:03 PM12/8/11
to dripls-dev
Ludo,

I have had no issues getting error codes in my modified m3u8's
(bitrate 404's, etc). The one issue that I seem to be having is:

File "/opt/dripls/dripls/shaper.py", line 172, in call_ext_shape_port
raise SystemError('Executing {0} failed with
{1}'.format(conf.shaper_path, p.returncode))
SystemError: Executing /opt/dripls/bin/set_ts_lo.sh failed with 1

I ensured that the apache user has password less sudo to the
set_ts_lo.sh script. I gave it a few test runs and I get the
following output:

$ sudo /opt/dripls/bin/set_ts_lo.sh 10000 100 10
RTNETLINK answers: File exists
RTNETLINK answers: File exists
------- Remove any previous rule
------- Adding rule

$ ls -lha /opt/dripls/bin
total 388K
drwxr-xr-x 2 root root 4.0K Dec 8 22:56 .
drwxr-xr-x 6 root root 4.0K Dec 7 11:44 ..
-rw-r--r-- 1 root root 223K Dec 8 22:59 access_log.log
-rwxr-xr-x 1 root root 3.0K Dec 6 18:16 dripls
-rw-rw-rw- 1 root root 4 Dec 8 22:56 dripls.pid
-rw-r--r-- 1 root root 133K Dec 8 22:59 error_log.log
-rwxr-xr-x 1 root root 1.2K Dec 8 21:04 set_ts_lo.sh

It's has the right perms, I also added the apache group to sudoers:

Cmnd_Alias NETWORK_SHAPING = /opt/dripls/bin/set_ts_lo.sh, /sbin/tc, /
sbin/iptables

## Allows members of the 'apache' group to run networking management
apps
%apache ALL=(ALL) NOPASSWD: NETWORK_SHAPING

Am I missing something? Bounced everything, including the host.

Thanks,

/GC

Ludo Antonov

unread,
Dec 8, 2011, 11:11:12 PM12/8/11
to dripl...@googlegroups.com
Hi Geoff,

hmm I wonder if it has something to do with the account you are running cherrypy with. Since apache just acts as a proxy to cherrypy, the account which runs the cherrypy dripls instance needs to have the right permissions. What account is cherrypy running with ?

Ludo

Geoff

unread,
Dec 8, 2011, 11:20:13 PM12/8/11
to dripls-dev
Cherrypy is running as root:

# ps -ef | grep dripls
root 5514 1 0 22:56 ? 00:00:05
dripls
root 5731 4342 0 23:17 pts/0 00:00:00 grep dripls

Unless there is somewhere else I should be looking to check this.

Ludo Antonov

unread,
Dec 8, 2011, 11:34:53 PM12/8/11
to dripl...@googlegroups.com
Bizarre . What distro are you running on ?

The output from set_ts_lo.sh seems correct. Could you try a couple of things for me ?

1) Try and move set_ts_lo.sh to set_ts_lo_old.sh and replacing it with just an empty script that doesn't do anything. And hit again to generate a shaped playlist and let me know if it still fails. This will isolate whether the issue is in the way the script is called or in the script itself. If i was to venture a guess it'd be the former.

2) In etc/sudoers , you should have a command

## Allow root to run any commands anywhere
root ALL=(ALL) ALL

I wonder if the sudo -n might have something to do with it ( in shaper.py) . If you remove the -n and sudo is indeed the problem, it should hang the process... this way we can filter out whether it's the sudo thats causing the problem.


Let me know what comes out of these two and we'll figure it out .

Ludo

Geoff

unread,
Dec 8, 2011, 11:39:42 PM12/8/11
to dripls-dev

On Dec 8, 11:34 pm, Ludo Antonov <ludo.anto...@gmail.com> wrote:
> Bizarre . What distro are you running on ?

Centos 5.5

>
> The output from set_ts_lo.sh seems correct. Could you try a couple of things for me  ?
>
> 1) Try and move set_ts_lo.sh to set_ts_lo_old.sh and replacing it with just an empty script that doesn't do anything. And hit again to generate a shaped playlist and let me know if it still fails. This will isolate whether the issue is in the way the script is called or in the script itself. If i was to venture a guess it'd be the former.
>

testing this now as well

> 2) In etc/sudoers , you should have a command
>
> ## Allow root to run any commands anywhere
> root    ALL=(ALL)       ALL

I do have this:


## Allow root to run any commands anywhere
root ALL=(ALL) ALL

>
> I wonder if the sudo -n might have something to do with it ( in shaper.py) . If you remove the -n and sudo is indeed the problem, it should hang the process... this way we can filter out whether it's the sudo thats causing the problem.

Testing now.

Ludo Antonov

unread,
Dec 8, 2011, 11:43:26 PM12/8/11
to dripl...@googlegroups.com
Ok , centos 5.5 should be ok. I've tested on it before.

Ludo

Geoff

unread,
Dec 8, 2011, 11:49:55 PM12/8/11
to dripls-dev
Ludo,

Using an empty script does nothing to change the behavior. When I
remove the -n, the behavior remains the same. As a side note, I am
able to do traffic shaping if I manually run tc and passing it
arguments, so iptables and netem appear to be working correctly. I do
see the following message in my syslog:

HTB: quantum of class 10001 is big. Consider r2q change.

which i believe has something to do with the first part of the
set_ts_lo.sh script. The other thing that I should mention, even
though it probably has no effect on the script, is that I am using
bonding on the interfaces.

/GC

Ludo Antonov

unread,
Dec 9, 2011, 12:01:35 AM12/9/11
to dripl...@googlegroups.com
Way bizarre. So then the problem is not with the set_ts_log, but rather with the Popen , it seems. So there is something to do with the way the command is called. Is sudo at /usr/bin/sudo ? could you try running : /usr/bin/sudo -n set_ts_lo.sh <with the parameters you had>

Also could you trace the log from :

shape_cmd = "sudo {0} {1} {2} {3}".format(conf.shaper_path, port, traffic_limit, traffic_loss)
logging.info("External shape call : {0} {1}".format(port, shape_cmd))

it should be in the cherrypy logs.

I don't think it has anything to do with the interfaces you are using. It shouldn't matter as it is only traffic shaping on the lo interface.


Ludo

Geoff

unread,
Dec 9, 2011, 12:21:38 AM12/9/11
to dripls-dev
Here is the output:

-bash-3.2$ whoami
apache
-bash-3.2$ /usr/bin/sudo -n /opt/dripls/bin/set_ts_lo.sh 10000 100 10


RTNETLINK answers: File exists
RTNETLINK answers: File exists
------- Remove any previous rule
------- Adding rule

And here is the cherrypy logging:

12/09/2011 12:16:49 AM DEBUG: matched rule : *.c1 in segment:
http://##scrubbed####ws.com/dripls/1700K/00/01.ts
12/09/2011 12:16:49 AM DEBUG: Segment cached http://##scrubbed####ws.com/dripls/1700K/00/01.ts
12/09/2011 12:16:49 AM INFO: External shape call : 10000 sudo /opt/
dripls/bin/set_ts_lo.sh 10000 5 0
12/09/2011 12:16:49 AM ERROR: [09/Dec/2011:00:16:49] HTTP Traceback
(most recent call last):
File "/usr/local/lib/python2.7/site-packages/cherrypy/
_cprequest.py", line 645, in respond
response.body = self.handler()
File "/usr/local/lib/python2.7/site-packages/cherrypy/lib/
encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/cherrypy/
_cpdispatch.py", line 29, in __call__
return self.callable(*self.args, **self.kwargs)
File "/opt/dripls/dripls/main.py", line 55, in master_m3u8
cached_cid = self.cache_stream(cid, r, tag, kwargs)["id"]
File "/opt/dripls/dripls/main.py", line 106, in cache_stream
info = shaper.cache_and_shape(master_playlist, seeded_content_id,
rules, master_playlist_url)
File "/opt/dripls/dripls/shaper.py", line 246, in cache_and_shape
seg_rewrite_url = segment_rule_rewrite(rules,
variant_playlist_desc, variant_playlist["segments"][s])
File "/opt/dripls/dripls/shaper.py", line 104, in
segment_rule_rewrite
return shape_segment(segment, rule_action,
mock_shape_segment=mock_shape_segment)
File "/opt/dripls/dripls/shaper.py", line 199, in shape_segment
call_ext_shape_port(port, traffic_limit, traffic_loss,
mock_shape_segment)
File "/opt/dripls/dripls/shaper.py", line 171, in


call_ext_shape_port
raise SystemError('Executing {0} failed with
{1}'.format(conf.shaper_path, p.returncode))
SystemError: Executing /opt/dripls/bin/set_ts_lo.sh failed with 1


/GC

Geoff

unread,
Dec 9, 2011, 12:25:35 AM12/9/11
to dripls-dev
And this is the URL:

curl "http://dripls01/master.m3u8?
authkey=sample&cid=url&cid_url=http://####scrubbed####ws.com/dripls/
master_rel.m3u8&r=*.c1~net5"


Ludo Antonov

unread,
Dec 9, 2011, 2:37:47 AM12/9/11
to dripl...@googlegroups.com
:( .. i am starting to run out of ides. Can we try one more thing . Can we change :

p = subprocess.Popen(["/usr/bin/sudo", "-n", conf.shaper_path, str(port), str(traffic_limit), str(traffic_loss)], stdin=subprocess.PIPE)
p.wait()

to

p = subprocess.Popen(["/bin/ls"], stdin=subprocess.PIPE)
p.wait()

and then:

p = subprocess.Popen(["/usr/bin/sudo", "/bin/ls"], stdin=subprocess.PIPE)
p.wait()

ideally getting the Popen to somehow succeed. And thus the request to complete without error. Once we get to that state, we can start fiddling with it and track back what is the specific thing that is causing the errors.


Ludo

Geoff

unread,
Dec 9, 2011, 3:48:42 AM12/9/11
to dripls-dev
So, a little bit of good news when I use just /bin/ls:

$ curl "http://dripls01/master.m3u8?
authkey=sample&cid=url&cid_url=http://####scrubbed####ws.com/dripls/
master_rel.m3u8&r=*.c1~net5,*.c2~net500.loss10"
#EXTM3U
#EXT-X-VERSION:2
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=4000000,RESOLUTION=960x540
http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477caeceb34_4000000__d
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1000000,RESOLUTION=960x540
http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477caeceb34_1000000__d
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1700000,RESOLUTION=960x540
http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477caeceb34_1700000__d
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=650000,RESOLUTION=960x540
http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477caeceb34_650000__d
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=250000,RESOLUTION=960x540
http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477caeceb34_250000__d

Annnnnnnnnd a little bit of bad news when I use sudo again:

$ curl "http://dripls01/master.m3u8?
authkey=sample&cid=url&cid_url=http://####scrubbed####ws.com/dripls/
master_rel.m3u8&r=*.c1~net5,*.c2~net500.loss10"
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8"></meta>
<title>500 Internal Server Error</title>
<style type="text/css">
#powered_by {
margin-top: 20px;
border-top: 2px solid black;
font-style: italic;
}

#traceback {
color: red;
}
</style>
</head>
<body>
<h2>500 Internal Server Error</h2>
<p>The server encountered an unexpected condition which
prevented it from fulfilling the request.</p>
<pre id="traceback">Traceback (most recent call last):


File "/usr/local/lib/python2.7/site-packages/cherrypy/
_cprequest.py", line 645, in respond
response.body = self.handler()
File "/usr/local/lib/python2.7/site-packages/cherrypy/lib/
encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/cherrypy/
_cpdispatch.py", line 29, in __call__
return self.callable(*self.args, **self.kwargs)
File "/opt/dripls/dripls/main.py", line 55, in master_m3u8
cached_cid = self.cache_stream(cid, r, tag, kwargs)["id"]
File "/opt/dripls/dripls/main.py", line 106, in cache_stream
info = shaper.cache_and_shape(master_playlist, seeded_content_id,
rules, master_playlist_url)

File "/opt/dripls/dripls/shaper.py", line 247, in cache_and_shape


seg_rewrite_url = segment_rule_rewrite(rules,
variant_playlist_desc, variant_playlist["segments"][s])
File "/opt/dripls/dripls/shaper.py", line 104, in
segment_rule_rewrite
return shape_segment(segment, rule_action,
mock_shape_segment=mock_shape_segment)

File "/opt/dripls/dripls/shaper.py", line 200, in shape_segment
call_ext_shape_port(port, traffic_limit, traffic_loss,
mock_shape_segment)
File "/opt/dripls/dripls/shaper.py", line 172, in


call_ext_shape_port
raise SystemError('Executing {0} failed with
{1}'.format(conf.shaper_path, p.returncode))
SystemError: Executing /opt/dripls/bin/set_ts_lo.sh failed with 1

</pre>
<div id="powered_by">
<span>Powered by <a href="http://www.cherrypy.org">CherryPy 3.2.0</
a></span>
</div>
</body>
</html>


Ludo Antonov

unread,
Dec 9, 2011, 3:56:10 AM12/9/11
to dripl...@googlegroups.com
Since cherrypy is already running as root , it might be acceptable to just change the sudo command for the set_ts_lo.sh command . This should in theory work. Let me know.
Message has been deleted

Ludo Antonov

unread,
Dec 9, 2011, 2:02:03 PM12/9/11
to dripl...@googlegroups.com
awesome! Yes, that should do it ... you can do something like :

~net3.loss20

and then do wget on the .ts segment and see it go really slow. That would be a good validation that it did.

Ludo


On Dec 9, 2011, at 1:04 AM, Geoff wrote:

> No sudo:


>
> $ curl "http://dripls01/master.m3u8?
> authkey=sample&cid=url&cid_url=http://####scrubbed####ws.com/dripls/
> master_rel.m3u8&r=*.c1~net5,*.c2~net500.loss10"
> #EXTM3U
> #EXT-X-VERSION:2
> #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=4000000,RESOLUTION=960x540

> http://dripls01.mm3.mlbam.com:8080/playlist.m3u8?p=m_url_035e4dd3cabf4a1e952d8826d8cc0684_4000000__d
> #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1000000,RESOLUTION=960x540
> http://dripls01.mm3.mlbam.com:8080/playlist.m3u8?p=m_url_035e4dd3cabf4a1e952d8826d8cc0684_1000000__d
> #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1700000,RESOLUTION=960x540
> http://dripls01.mm3.mlbam.com:8080/playlist.m3u8?p=m_url_035e4dd3cabf4a1e952d8826d8cc0684_1700000__d
> #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=650000,RESOLUTION=960x540
> http://dripls01.mm3.mlbam.com:8080/playlist.m3u8?p=m_url_035e4dd3cabf4a1e952d8826d8cc0684_650000__d
> #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=250000,RESOLUTION=960x540
> http://dripls01.mm3.mlbam.com:8080/playlist.m3u8?p=m_url_035e4dd3cabf4a1e952d8826d8cc0684_250000__d
>
>
> I am assuming at this point that the manipulation was completed, but I
> have nothing to test it with at the moment because I am home. I will
> let you know tomorrow evening, but I think we will be calling this one
> solved. Have a good night Ludo.
>
> /GC


>
>
> On Dec 9, 3:56 am, Ludo Antonov <ludo.anto...@gmail.com> wrote:
>> Since cherrypy is already running as root , it might be acceptable to just change the sudo command for the set_ts_lo.sh command . This should in theory work. Let me know.
>>
>> On Dec 9, 2011, at 12:48 AM, Geoff wrote:
>>
>>
>>
>>
>>
>>
>>
>>> So, a little bit of good news when I use just /bin/ls:
>>
>>> $ curl "http://dripls01/master.m3u8?
>>> authkey=sample&cid=url&cid_url=http://####scrubbed####ws.com/dripls/
>>> master_rel.m3u8&r=*.c1~net5,*.c2~net500.loss10"
>>> #EXTM3U
>>> #EXT-X-VERSION:2
>>> #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=4000000,RESOLUTION=960x540

>>> http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477ca...
>>> #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1000000,RESOLUTION=960x540
>>> http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477ca...
>>> #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1700000,RESOLUTION=960x540
>>> http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477ca...
>>> #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=650000,RESOLUTION=960x540
>>> http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477ca...
>>> #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=250000,RESOLUTION=960x540
>>> http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477ca...

Geoff

unread,
Dec 9, 2011, 5:59:46 PM12/9/11
to dripls-dev
$ curl "http://dripls01/master.m3u8?
authkey=sample&cid=url&cid_url=http://####scrubbedd####ws.com/dripls/
master_rel.m3u8&r=*.c1~net5"

#EXTM3U
#EXT-X-VERSION:2
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=4000000,RESOLUTION=960x540
http://dripls01:8080/playlist.m3u8?p=m_url_5f05b30d0cb94b82856158e9fe49fc4e_4000000__d
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1000000,RESOLUTION=960x540
http://dripls01:8080/playlist.m3u8?p=m_url_5f05b30d0cb94b82856158e9fe49fc4e_1000000__d
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1700000,RESOLUTION=960x540
http://dripls01:8080/playlist.m3u8?p=m_url_5f05b30d0cb94b82856158e9fe49fc4e_1700000__d
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=650000,RESOLUTION=960x540
http://dripls01:8080/playlist.m3u8?p=m_url_5f05b30d0cb94b82856158e9fe49fc4e_650000__d
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=250000,RESOLUTION=960x540
http://dripls01:8080/playlist.m3u8?p=m_url_5f05b30d0cb94b82856158e9fe49fc4e_250000__d


On Dec 9, 3:56 am, Ludo Antonov <ludo.anto...@gmail.com> wrote:

> Since cherrypy is already running as root , it might be acceptable to just change the sudo command for the set_ts_lo.sh command . This should in theory work. Let me know.
>
> On Dec 9, 2011, at 12:48 AM, Geoff wrote:
>
>
>
>
>
>
>
> > So, a little bit of good news when I use just /bin/ls:
>
> > $ curl "http://dripls01/master.m3u8?
> > authkey=sample&cid=url&cid_url=http://####scrubbed####ws.com/dripls/
> > master_rel.m3u8&r=*.c1~net5,*.c2~net500.loss10"
> > #EXTM3U
> > #EXT-X-VERSION:2
> > #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=4000000,RESOLUTION=960x540

> >http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477ca...
> > #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1000000,RESOLUTION=960x540
> >http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477ca...
> > #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1700000,RESOLUTION=960x540
> >http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477ca...
> > #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=650000,RESOLUTION=960x540
> >http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477ca...
> > #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=250000,RESOLUTION=960x540
> >http://dripls01:8080/playlist.m3u8?p=m_url_771a03360aa6471184eb1477ca...
>

Ludo Antonov

unread,
Dec 9, 2011, 6:28:54 PM12/9/11
to dripl...@googlegroups.com
Hi Geoff,
i just got a playlist :) .. is there any problem with it ?

Ludo

Reply all
Reply to author
Forward
0 new messages