Re: [saga-users:458] submitting jobs to queue on alamo

[Mark Santcroos]

Hi Ajinkya,

pbs:// is not a valid pd_service_url as you can’t store data into “pbs”.
In case of Alamo, you want to use ssh:// to store your data in a PD.

I actually ran into the same error yesterday (for a different reason though) and have made a change to BigJob to give a more meaningful error.

Gr,

Mark


On 05 Dec 2013, at 4:15 , Ajinkya Bilolikar <ajinkya.b...@gmail.com> wrote:

> Hello,
> I am running a local implementation of PilotMapReduce on Alamo. The issue I am facing is with submitting my jobs to the queue. The part of the scrip which uses the saga adaptor is as follows:
>
> "pj_service_url": 'fork://localhost',
> "pd_service_url":"ssh://localhost/"+os.getcwd()+"/pilotdata",
>
> Here I would like to use pbs in addition to fork and ssh. The error I am getting after replacing the above pj_service_url with pbs://localhost and pd_service_url with pbs://localhost+os.getcwd()+"/pilotdata",is
>
> File "single_WC.py", line 42, in <module>
> mr.MapReduceMain()
> File "/N/u/nv121/aj1/lib/python2.7/site-packages/PilotMapReduce-0.1.0-py2.7.egg/pmr/PilotMapReduce.py", line 370, in MapReduceMain
> self.start_pilot_datas()
> File "/N/u/nv121/aj1/lib/python2.7/site-packages/PilotMapReduce-0.1.0-py2.7.egg/pmr/PilotMapReduce.py", line 69, in start_pilot_datas
> self.pilot_data_service.create_pilot( pilot_data_description=pd_desc )
> File "/N/u/nv121/aj1/lib/python2.7/site-packages/BigJob-0.52-py2.7.egg/pilot/impl/pilotdata_manager.py", line 367, in create_pilot
> pilot_data_description=pilot_data_description)
> File "/N/u/nv121/aj1/lib/python2.7/site-packages/BigJob-0.52-py2.7.egg/pilot/impl/pilotdata_manager.py", line 124, in __init__
> self.__initialize_pilot_data()
> File "/N/u/nv121/aj1/lib/python2.7/site-packages/BigJob-0.52-py2.7.egg/pilot/impl/pilotdata_manager.py", line 274, in __initialize_pilot_data
> raise PilotError("No File Plugin found.")
> pilot.api.api.PilotError: 'No File Plugin found.'
>
> Can some one please suggest me the appropriate changes to be made.
>
> Regards
>
> Ajinkya
>
>
> --
> You received this message because you are subscribed to the Google Groups "saga-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to saga-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Ajinkya Bilolikar]

Hello,

         Like you suggested I changed to "pd_service_url":"pbs+ssh://localhost/"+os.getcwd()+"/pilotdata", I have not made any changes to pj_service_url. The reason I need pbs and not just ssh is because I want my jobs to run on the queue. However using pbs+ ssh still gives me the same error which I displayed in my previous email.

        

        My understanding (which is not a lot) is that we use ssh only for logging into a remote machine. In this case I am running my jobs on the same machine(alamo). Would "pbs+ssh" work in such a scenario?

Thank You

Ajinkya

On Thu, Dec 5, 2013 at 12:31 PM, Ajinkya Bilolikar <ajinkya.b...@gmail.com> wrote:

Hello,

         Like you suggested I changed to "pd_service_url":"pbs+ssh://localhost/"+os.getcwd()+"/pilotdata", I have not made any changes to pj_service_url. The reason I need pbs and not just ssh is because I want my jobs to run on the queue. However using pbs+ ssh still gives me the same error which I displayed in my previous email.

        

        My understanding (which is not a lot) is that we use ssh only for logging into a remote machine. In this case I am running my jobs on the same machine(alamo). Would "pbs+ssh" work in such a scenario?

Thank You

Ajinkya

On Thu, Dec 5, 2013 at 10:43 AM, Ajinkya Bilolikar <ajinkya.b...@gmail.com> wrote:

Hello,

         Like you suggested I changed to "pd_service_url":"pbs+ssh://localhost/"+os.getcwd()+"/pilotdata", I have not made any changes to pj_service_url. The reason I need pbs and not just ssh is because I want my jobs to run on the queue. However using pbs+ ssh still gives me the same error which I displayed in my previous email.

        

        My understanding (which is not a lot) is that we use ssh only forlogging into a remote machine. In this case I am running my jobs on the same machine(alamo). Would "pbs+ssh" work in such a scenario?

Thank You

Ajinkya

You received this message because you are subscribed to a topic in the Google Groups "saga-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/saga-users/Njry7B5-zU4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to saga-users+...@googlegroups.com.

[Mark Santcroos]

Hi,

On 06 Dec 2013, at 23:34 , Ajinkya Bilolikar <ajinkya.b...@gmail.com> wrote:
> Like you suggested I changed to "pd_service_url":"pbs+ssh://localhost/"+os.getcwd()+"/pilotdata", I have not made any changes to pj_service_url. The reason I need pbs and not just ssh is because I want my jobs to run on the queue. However using pbs+ ssh still gives me the same error which I displayed in my previous email.

I dont think that was what I suggested :-)

"In case of Alamo, you want to use ssh:// to store your data in a PD.”

> My understanding (which is not a lot) is that we use ssh only for logging into a remote machine. In this case I am running my jobs on the same machine(alamo). Would "pbs+ssh" work in such a scenario?

Yes, although you need to make sure that you can ssh into localhost first (adding public key to .ssh/authorized_keys for example).

Gr,

Mark

> You received this message because you are subscribed to the Google Groups "bigjob-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bigjob-users...@googlegroups.com.
> To post to this group, send email to bigjob...@googlegroups.com.
> Visit this group at http://groups.google.com/group/bigjob-users.