I am trying to setup an AMI to run boto's "sonofmmm" service. I have managed to do the following so far: * Created input and output queues, buckets and sdb domain, * Launched a new small instance of https://console.aws.amazon.com/ec2/home?region=eu-west-1#launchAmi=am... AMI, * Installed ffmpeg, * Git cloned latest (2.3.0) boto source and installed it, * Put the following into rc.local:
once the new instance is launched I get the following error in syslog:
> Traceback (most recent call last): > File "/home/<username>/boto/boto/pyami/startup.py", line 60, in <module> > su.main() > File "/home/<username>/boto/boto/pyami/startup.py", line 52, in main > self.run_scripts() > File "/home/<username>/boto/boto/pyami/startup.py", line 49, in > run_scripts > raise e > AttributeError: 'NoneType' object has no attribute 'read'
I have also logged in via ssh to this instance and looked at boto's log files, look at the boto source code and it seems that this issue has to do with the default region that is set for the input queue (sqs). It is set to "us-east-1" and I can't seem to find a proper way to set region for aws services to "eu-west-1". Setting "ec2_region_name" and "ec2_region_endpoint" in the service config file doesn't seem to help.
It informs me that all the files were submitted and prints the "Batch Identifier". I can confirm that these files are stored in the input bucket but no new messages are created in the input queue.
This may or may not be the "right" answer (in reality, it's a shameless
plug), but I could suggest checking out media-nommer if you can't figure
sonofmm out:
On Tue, May 8, 2012 at 4:17 PM, nikola <nik...@imadethis.no> wrote:
> Hello,
> I am trying to setup an AMI to run boto's "sonofmmm" service. I have
> managed to do the following so far:
> * Created input and output queues, buckets and sdb domain,
> * Launched a new small instance of
> https://console.aws.amazon.com/ec2/home?region=eu-west-1#launchAmi=am...,
> * Installed ffmpeg,
> * Git cloned latest (2.3.0) boto source and installed it,
> * Put the following into rc.local:
> once the new instance is launched I get the following error in syslog:
>> Traceback (most recent call last):
>> File "/home/<username>/boto/boto/pyami/startup.py", line 60, in <module>
>> su.main()
>> File "/home/<username>/boto/boto/pyami/startup.py", line 52, in main
>> self.run_scripts()
>> File "/home/<username>/boto/boto/pyami/startup.py", line 49, in
>> run_scripts
>> raise e
>> AttributeError: 'NoneType' object has no attribute 'read'
> I have also logged in via ssh to this instance and looked at boto's log
> files, look at the boto source code and it seems that this issue has to do
> with the default region that is set for the input queue (sqs). It is set to
> "us-east-1" and I can't seem to find a proper way to set region for aws
> services to "eu-west-1".
> Setting "ec2_region_name" and "ec2_region_endpoint" in the service config
> file doesn't seem to help.
> It informs me that all the files were submitted and prints the "Batch
> Identifier". I can confirm that these files are stored in the input bucket
> but no new messages are created in the input queue.
> Any help would be appreciated.
> Kind Regards,
> Nikola
> --
> You received this message because you are subscribed to the Google Groups
> "boto-users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/boto-users/-/beafXGkBFbcJ.
> To post to this group, send email to boto-users@googlegroups.com.
> To unsubscribe from this group, send email to
> boto-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/boto-users?hl=en.
I am looking for a way to create a distributed application in the AWS cloud that could launch worker instances that will take input A and process it and produce output B, in other words "get the job(s) done". My architecture should be similar to the on this image<http://awsmedia.s3.amazonaws.com/catalog/images/411-diagram.jpg>. I chose boto and "sonofmmm" service as a starting point because I found that it provides exactly what I am looking for. My idea was to get this running and then write my own "sonofmmm"-like service to perform custom jobs. It actually doesn't have to do anything with converting video files.
Nevertheless, I am going to look at the media-nommer as it seems very interesting.
On Tuesday, May 8, 2012 10:43:16 PM UTC+2, Gregory Taylor wrote:
> This may or may not be the "right" answer (in reality, it's a shameless > plug), but I could suggest checking out media-nommer if you can't figure > sonofmm out:
>> I am trying to setup an AMI to run boto's "sonofmmm" service. I have >> managed to do the following so far: >> * Created input and output queues, buckets and sdb domain, >> * Launched a new small instance of >> https://console.aws.amazon.com/ec2/home?region=eu-west-1#launchAmi=am..., >> * Installed ffmpeg, >> * Git cloned latest (2.3.0) boto source and installed it, >> * Put the following into rc.local:
>> once the new instance is launched I get the following error in syslog:
>>> Traceback (most recent call last): >>> File "/home/<username>/boto/boto/pyami/startup.py", line 60, in >>> <module> >>> su.main() >>> File "/home/<username>/boto/boto/pyami/startup.py", line 52, in main >>> self.run_scripts() >>> File "/home/<username>/boto/boto/pyami/startup.py", line 49, in >>> run_scripts >>> raise e >>> AttributeError: 'NoneType' object has no attribute 'read'
>> I have also logged in via ssh to this instance and looked at boto's log >> files, look at the boto source code and it seems that this issue has to do >> with the default region that is set for the input queue (sqs). It is set to >> "us-east-1" and I can't seem to find a proper way to set region for aws >> services to "eu-west-1". >> Setting "ec2_region_name" and "ec2_region_endpoint" in the service config >> file doesn't seem to help.
>> It informs me that all the files were submitted and prints the "Batch >> Identifier". I can confirm that these files are stored in the input bucket >> but no new messages are created in the input queue.
>> Any help would be appreciated.
>> Kind Regards, >> Nikola
>> -- >> You received this message because you are subscribed to the Google Groups >> "boto-users" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/boto-users/-/beafXGkBFbcJ. >> To post to this group, send email to boto-users@googlegroups.com. >> To unsubscribe from this group, send email to >> boto-users+unsubscribe@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/boto-users?hl=en.
I think the basic pattern embodied in SonOfMMM is still extremely useful
but the code is old. I think we should either agree there are better, more
modern alternatives like media-nommer and try to point people who read the
SonOfMMM article at those alternatives or spend the time to update the
SonOfMMM ami and code. If the best answer is media-nommer (entirely
possible) then perhaps we could even get a new article published that
focuses on solving the same basic problem as the SonOfMMM article using
media-nommer.
On Tue, May 8, 2012 at 2:07 PM, nikola <nik...@imadethis.no> wrote:
> Hello Greg,
> thanks for your quick reply.
> I am looking for a way to create a distributed application in the AWS
> cloud that could launch worker instances that will take input A and process
> it and produce output B, in other words "get the job(s) done". My
> architecture should be similar to the on this image<http://awsmedia.s3.amazonaws.com/catalog/images/411-diagram.jpg>.
> I chose boto and "sonofmmm" service as a starting point because I found
> that it provides exactly what I am looking for. My idea was to get this
> running and then write my own "sonofmmm"-like service to perform custom
> jobs. It actually doesn't have to do anything with converting video files.
> Nevertheless, I am going to look at the media-nommer as it seems very
> interesting.
> Thanks,
> Nikola
> On Tuesday, May 8, 2012 10:43:16 PM UTC+2, Gregory Taylor wrote:
>> This may or may not be the "right" answer (in reality, it's a shameless
>> plug), but I could suggest checking out media-nommer if you can't figure
>> sonofmm out:
>>> I am trying to setup an AMI to run boto's "sonofmmm" service. I have
>>> managed to do the following so far:
>>> * Created input and output queues, buckets and sdb domain,
>>> * Launched a new small instance of https://console.aws.amazon.** >>> com/ec2/home?region=eu-west-1#**launchAmi=ami-1de8d369<https://console.aws.amazon.com/ec2/home?region=eu-west-1#launchAmi=am...>AMI,
>>> * Installed ffmpeg,
>>> * Git cloned latest (2.3.0) boto source and installed it,
>>> * Put the following into rc.local:
>>> once the new instance is launched I get the following error in syslog:
>>>> Traceback (most recent call last):
>>>> File "/home/<username>/boto/boto/**pyami/startup.py", line 60, in
>>>> <module>
>>>> su.main()
>>>> File "/home/<username>/boto/boto/**pyami/startup.py", line 52, in
>>>> main
>>>> self.run_scripts()
>>>> File "/home/<username>/boto/boto/**pyami/startup.py", line 49, in
>>>> run_scripts
>>>> raise e
>>>> AttributeError: 'NoneType' object has no attribute 'read'
>>> I have also logged in via ssh to this instance and looked at boto's log
>>> files, look at the boto source code and it seems that this issue has to do
>>> with the default region that is set for the input queue (sqs). It is set to
>>> "us-east-1" and I can't seem to find a proper way to set region for aws
>>> services to "eu-west-1".
>>> Setting "ec2_region_name" and "ec2_region_endpoint" in the service
>>> config file doesn't seem to help.
>>> It informs me that all the files were submitted and prints the "Batch
>>> Identifier". I can confirm that these files are stored in the input bucket
>>> but no new messages are created in the input queue.
>>> Any help would be appreciated.
>>> Kind Regards,
>>> Nikola
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "boto-users" group.
>>> To view this discussion on the web visit https://groups.google.com/d/** >>> msg/boto-users/-/beafXGkBFbcJ<https://groups.google.com/d/msg/boto-users/-/beafXGkBFbcJ>
>>> .
>>> To post to this group, send email to boto-users@googlegroups.com.
>>> To unsubscribe from this group, send email to boto-users+unsubscribe@**
>>> googlegroups.com <boto-users%2Bunsubscribe@googlegroups.com>.
>>> For more options, visit this group at http://groups.google.com/** >>> group/boto-users?hl=en <http://groups.google.com/group/boto-users?hl=en>
>>> .
> To post to this group, send email to boto-users@googlegroups.com.
> To unsubscribe from this group, send email to
> boto-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/boto-users?hl=en.
I did some quick googling and found this<http://www.ubuntu-user.com/Online/Blogs/Amber-Graner-You-in-Ubuntu/Gr...>article where I read that I could accomplish my goal by writing a custom "Nommer". I guess your reply just verified this. It would be of great help to have some sort of tutorial-like article.
I spent a lot of time on getting to know and setting up SonOfMMM. I am wondering if it would be more feasible to switch to media-nommer or to pursue with the former one?
On Tuesday, May 8, 2012 11:21:24 PM UTC+2, Mitchell Garnaat wrote:
> I think the basic pattern embodied in SonOfMMM is still extremely useful > but the code is old. I think we should either agree there are better, more > modern alternatives like media-nommer and try to point people who read the > SonOfMMM article at those alternatives or spend the time to update the > SonOfMMM ami and code. If the best answer is media-nommer (entirely > possible) then perhaps we could even get a new article published that > focuses on solving the same basic problem as the SonOfMMM article using > media-nommer.
> On Tue, May 8, 2012 at 2:07 PM, nikola wrote:
>> Hello Greg,
>> thanks for your quick reply.
>> I am looking for a way to create a distributed application in the AWS >> cloud that could launch worker instances that will take input A and process >> it and produce output B, in other words "get the job(s) done". My >> architecture should be similar to the on this image<http://awsmedia.s3.amazonaws.com/catalog/images/411-diagram.jpg>. >> I chose boto and "sonofmmm" service as a starting point because I found >> that it provides exactly what I am looking for. My idea was to get this >> running and then write my own "sonofmmm"-like service to perform custom >> jobs. It actually doesn't have to do anything with converting video files.
>> Nevertheless, I am going to look at the media-nommer as it seems very >> interesting.
>> Thanks, >> Nikola
>> On Tuesday, May 8, 2012 10:43:16 PM UTC+2, Gregory Taylor wrote:
>>> This may or may not be the "right" answer (in reality, it's a shameless >>> plug), but I could suggest checking out media-nommer if you can't figure >>> sonofmm out:
>>>> I am trying to setup an AMI to run boto's "sonofmmm" service. I have >>>> managed to do the following so far: >>>> * Created input and output queues, buckets and sdb domain, >>>> * Launched a new small instance of https://console.aws.amazon.** >>>> com/ec2/home?region=eu-west-1#**launchAmi=ami-1de8d369<https://console.aws.amazon.com/ec2/home?region=eu-west-1#launchAmi=am...>AMI, >>>> * Installed ffmpeg, >>>> * Git cloned latest (2.3.0) boto source and installed it, >>>> * Put the following into rc.local:
>>>> once the new instance is launched I get the following error in syslog:
>>>>> Traceback (most recent call last): >>>>> File "/home/<username>/boto/boto/**pyami/startup.py", line 60, in >>>>> <module> >>>>> su.main() >>>>> File "/home/<username>/boto/boto/**pyami/startup.py", line 52, in >>>>> main >>>>> self.run_scripts() >>>>> File "/home/<username>/boto/boto/**pyami/startup.py", line 49, in >>>>> run_scripts >>>>> raise e >>>>> AttributeError: 'NoneType' object has no attribute 'read'
>>>> I have also logged in via ssh to this instance and looked at boto's log >>>> files, look at the boto source code and it seems that this issue has to do >>>> with the default region that is set for the input queue (sqs). It is set to >>>> "us-east-1" and I can't seem to find a proper way to set region for aws >>>> services to "eu-west-1". >>>> Setting "ec2_region_name" and "ec2_region_endpoint" in the service >>>> config file doesn't seem to help.
>>>> It informs me that all the files were submitted and prints the "Batch >>>> Identifier". I can confirm that these files are stored in the input bucket >>>> but no new messages are created in the input queue.
>>>> Any help would be appreciated.
>>>> Kind Regards, >>>> Nikola
>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "boto-users" group. >>>> To view this discussion on the web visit https://groups.google.com/d/** >>>> msg/boto-users/-/beafXGkBFbcJ<https://groups.google.com/d/msg/boto-users/-/beafXGkBFbcJ> >>>> . >>>> To post to this group, send email to boto-users@googlegroups.com. >>>> To unsubscribe from this group, send email to boto-users+unsubscribe@** >>>> googlegroups.com <boto-users%2Bunsubscribe@googlegroups.com>. >>>> For more options, visit this group at http://groups.google.com/** >>>> group/boto-users?hl=en<http://groups.google.com/group/boto-users?hl=en> >>>> .
>> To post to this group, send email to boto-users@googlegroups.com. >> To unsubscribe from this group, send email to >> boto-users+unsubscribe@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/boto-users?hl=en.
On Tue, May 8, 2012 at 5:54 PM, nikola <nik...@imadethis.no> wrote:
> I did some quick googling and found this<http://www.ubuntu-user.com/Online/Blogs/Amber-Graner-You-in-Ubuntu/Gr...>article where I read that I could accomplish my goal by writing a custom
> "Nommer". I guess your reply just verified this. It would be of great help
> to have some sort of tutorial-like article.
I do need to write something like that. I'll try to get something figured
out in the near future.
> I spent a lot of time on getting to know and setting up SonOfMMM. I am
> wondering if it would be more feasible to switch to media-nommer or
> to pursue with the former one?
I'm not sure there. As Mitch has mentioned, it is pretty old, and doesn't
see a lot of attention. However, if you're already familiar with it, and
can get it working, it may not be worth getting to learn yet another
project. I don't want to direct you the wrong way, there. You definitely
would need to write a Nommer for whatever you're doing. While it's not very
difficult, it is another step.
If you do end up having more questions about media-nommer, shoot me an
email at gtay...@gc-taylor.com so I won't annoy boto-users. Otherwise, keep
on keeping on with SonOfMMM!
On Wednesday, May 9, 2012 12:01:08 AM UTC+2, Gregory Taylor wrote:
> On Tue, May 8, 2012 at 5:54 PM, nikola wrote:
>> I did some quick googling and found this<http://www.ubuntu-user.com/Online/Blogs/Amber-Graner-You-in-Ubuntu/Gr...>article where I read that I could accomplish my goal by writing a custom >> "Nommer". I guess your reply just verified this. It would be of great help >> to have some sort of tutorial-like article.
> I do need to write something like that. I'll try to get something figured > out in the near future.
>> I spent a lot of time on getting to know and setting up SonOfMMM. I am >> wondering if it would be more feasible to switch to media-nommer or >> to pursue with the former one?
> I'm not sure there. As Mitch has mentioned, it is pretty old, and doesn't > see a lot of attention. However, if you're already familiar with it, and > can get it working, it may not be worth getting to learn yet another > project. I don't want to direct you the wrong way, there. You definitely > would need to write a Nommer for whatever you're doing. While it's not very > difficult, it is another step.
> If you do end up having more questions about media-nommer, shoot me an > email at gtay...@gc-taylor.com so I won't annoy boto-users. Otherwise, > keep on keeping on with SonOfMMM!
