[saga-project/BigJob] a9ac32: examples should not have syntax errors, in particu...
0 views
Skip to first unread message
Pradeep Mantha
Mar 14, 2026, 1:24:38 PMMar 14
to bigjob...@googlegroups.com
Branch: refs/heads/registerExistingDUFIles
Home: https://github.com/saga-project/BigJob
Commit: a9ac32f82abc70df2c3180328785e2dde417c0ab
https://github.com/saga-project/BigJob/commit/a9ac32f82abc70df2c3180328785e2dde417c0ab
Author: Andre Merzky <an...@merzky.net>
Date: 2014-01-06 (Mon, 06 Jan 2014)
Changed paths:
M examples/tutorial/simple_ensembles.py
Log Message:
-----------
examples should not have syntax errors, in particular if they are installed...
Commit: c86764b62909e3290a24054af87b0017784e92ed
https://github.com/saga-project/BigJob/commit/c86764b62909e3290a24054af87b0017784e92ed
Author: Andre Merzky <an...@merzky.net>
Date: 2014-01-06 (Mon, 06 Jan 2014)
Changed paths:
M examples/tutorial/barebones-local/local_chained_ensembles.py
M examples/tutorial/barebones-local/local_coupled_ensembles.py
M examples/tutorial/barebones-local/local_simple_ensembles.py
M examples/tutorial/chained_ensembles.py
M examples/tutorial/coupled_ensembles.py
M examples/tutorial/simple_ensemble_dataxfer.py
M examples/tutorial/simple_ensembles.py
Log Message:
-----------
examples should not have syntax errors, in particular if they are installed... And tabs are bad, too!
Commit: 4b41cc6ea7c97caa8fcede29f19c94c0be80c7c7
https://github.com/saga-project/BigJob/commit/4b41cc6ea7c97caa8fcede29f19c94c0be80c7c7
Author: Mark Santcroos <mas...@scarletmail.rutgers.edu>
Date: 2013-12-05 (Thu, 05 Dec 2013)
Changed paths:
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
Report the service_url in the exception to assist debugging.
Commit: adea1098cca668e864aa2722661898ab61b100eb
https://github.com/saga-project/BigJob/commit/adea1098cca668e864aa2722661898ab61b100eb
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-05 (Thu, 05 Dec 2013)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob_dynamic/many_job.py
M bootstrap/bigjob-bootstrap.py
M docs/source/conf.py
M docs/source/patterns/chained.rst
R docs/source/usage/appwriting.rst
M docs/source/usage/index.rst
R docs/source/usage/pilotdata.rst
M examples/tutorial/simple_ensembles.py
M pilot/impl/pilotcompute_manager.py
Log Message:
-----------
Merge branch 'master' into develop
# By Melissa Romanus (6) and Andre Luckow (2)
# Via Melissa Romanus
* master:
Fix indices
Remove appwriting and pilotdata, these should not be front-facing pages anymore
upgraded BJ to latest virtualenv/distribute
tabs vs. spaces
Using explicit spaces
PCD needs to take a None string for new more expressive examples to work
BigJob moves to cleaner theme. Will no longer match saga-python
Fix issue 158 for mturilli
Commit: 69235c30dcb80d3b82a95c41e3129217856dfe51
https://github.com/saga-project/BigJob/commit/69235c30dcb80d3b82a95c41e3129217856dfe51
Author: Mark Santcroos <mas...@scarletmail.rutgers.edu>
Date: 2013-12-06 (Fri, 06 Dec 2013)
Changed paths:
M bigjob/bigjob_manager.py
Log Message:
-----------
Convert and correct spurious print into a debug message.
Commit: fadc00a445a5474be795c3276cd1417f0ff515d2
https://github.com/saga-project/BigJob/commit/fadc00a445a5474be795c3276cd1417f0ff515d2
Author: Mark Santcroos <mas...@scarletmail.rutgers.edu>
Date: 2013-12-06 (Fri, 06 Dec 2013)
Changed paths:
M pilot/filemanagement/ssh_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
Create parent dirs also, for both ssh and local.
Commit: a3613787d81d1d7954f4daa9ddd9bbf97dcb971f
https://github.com/saga-project/BigJob/commit/a3613787d81d1d7954f4daa9ddd9bbf97dcb971f
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-18 (Wed, 18 Dec 2013)
Changed paths:
M bigjob/bigjob_manager.py
M pilot/filemanagement/ssh_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
Merge branch 'develop' of github.com:saga-project/BigJob into develop
Commit: 473f8fed78bd9ec3845fe5f56d0a74b8e4a43ef8
https://github.com/saga-project/BigJob/commit/473f8fed78bd9ec3845fe5f56d0a74b8e4a43ef8
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-18 (Wed, 18 Dec 2013)
Changed paths:
M bigjob.conf
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
A bigjob/job_plugin/mesos/README
A bigjob/job_plugin/mesos/__init__.py
A bigjob/job_plugin/mesos/mesos_executor
A bigjob/job_plugin/mesos/mesos_executor.py
A bigjob/job_plugin/mesos/mesos_job.py
R distribute_setup.py
A examples/bigdata/example-pilot-compute-mesos.py
M setup.py
Log Message:
-----------
Initial support for Mesos scheduler
Commit: f5c695fe8c441d64ab96cce47bbf3a4c24f05bbe
https://github.com/saga-project/BigJob/commit/f5c695fe8c441d64ab96cce47bbf3a4c24f05bbe
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
R bigjob/job_plugin/mesos/README
M bigjob/job_plugin/mesos/mesos_job.py
A bigjob/job_plugin/yarn/README.md
A bigjob/job_plugin/yarn/__init__.py
A bigjob/job_plugin/yarn/yarn.py
A bootstrap/bigjob-bootstrap.sh
A examples/bigdata/example-pilot-compute-yarn.py
Log Message:
-----------
Work on YARN plugin
Commit: 735d57dfe054290276e7fcc3c674f40654fba78b
https://github.com/saga-project/BigJob/commit/735d57dfe054290276e7fcc3c674f40654fba78b
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/job_plugin/yarn/yarn.py
R bootstrap/bigjob-bootstrap.sh
A bootstrap/bigjob2-bootstrap.py
A bootstrap/bigjob2-bootstrap.sh
M setup.py
Log Message:
-----------
added new PyPi version to test new Bootstrapping method
Commit: 2d1153c4753aeb9362a27403ad21768431479be2
https://github.com/saga-project/BigJob/commit/2d1153c4753aeb9362a27403ad21768431479be2
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/yarn.py
M bootstrap/bigjob2-bootstrap.sh
Log Message:
-----------
minor tweaks to yarn
Commit: 8d8aa283ab5351333de6a4f4639d5874b8643590
https://github.com/saga-project/BigJob/commit/8d8aa283ab5351333de6a4f4639d5874b8643590
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M MANIFEST.in
Log Message:
-----------
update MANIFESt
Commit: c4150133fa2de9f0bef061b775d6ce35e287cf38
https://github.com/saga-project/BigJob/commit/c4150133fa2de9f0bef061b775d6ce35e287cf38
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M bootstrap/bigjob2-bootstrap.py
M bootstrap/generate_bigjob_bootstrap_script.py
Log Message:
-----------
refined bigjob2 bootstrap
Commit: 1f1633bdc0cc20da65b4c3142ef06df05450fcf9
https://github.com/saga-project/BigJob/commit/1f1633bdc0cc20da65b4c3142ef06df05450fcf9
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/yarn.py
M bootstrap/bigjob2-bootstrap.py
Log Message:
-----------
fixed bootstrap for YARN
Commit: 6ca38f7c2fd4f346844b34b743c561136f9acbeb
https://github.com/saga-project/BigJob/commit/6ca38f7c2fd4f346844b34b743c561136f9acbeb
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
M bigjob/job_plugin/mesos/mesos_job.py
M bigjob/job_plugin/yarn/yarn.py
M bootstrap/bigjob2-bootstrap.sh
R examples/bigdata/example-pilot-compute-mesos.py
R examples/bigdata/example-pilot-compute-yarn.py
Log Message:
-----------
further work on YARN adaptor and bootstrapping
Commit: 8fdbd7b0b03db2776d05bba4589e2d4e557a4f43
https://github.com/saga-project/BigJob/commit/8fdbd7b0b03db2776d05bba4589e2d4e557a4f43
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
A examples/hadoop/example-pilot-compute-mesos.py
A examples/hadoop/example-pilot-compute-yarn.py
Log Message:
-----------
further work on YARN adaptor and bootstrapping
Commit: 0294d27694187fa51675497997385a562de41ecd
https://github.com/saga-project/BigJob/commit/0294d27694187fa51675497997385a562de41ecd
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/yarn.py
Log Message:
-----------
removed debugging output
Commit: 913d633b7a8b848ce4ccd43000ab8786a5e7ca2e
https://github.com/saga-project/BigJob/commit/913d633b7a8b848ce4ccd43000ab8786a5e7ca2e
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
A bigjob/job_plugin/yarn/BigJob-YARN-0.1-SNAPSHOT-jar-with-dependencies.jar
M bigjob/job_plugin/yarn/yarn.py
Log Message:
-----------
for simplicity added YARN BigJob jar
Commit: dbfe26c753e55d54674c58cf238c2fd399596e87
https://github.com/saga-project/BigJob/commit/dbfe26c753e55d54674c58cf238c2fd399596e87
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/yarn.py
M examples/hadoop/example-pilot-compute-yarn.py
Log Message:
-----------
fixes
Commit: 375031788c44edb6726b20f91afb69e2f41e745d
https://github.com/saga-project/BigJob/commit/375031788c44edb6726b20f91afb69e2f41e745d
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/README.md
Log Message:
-----------
readme expanded
Commit: 4f6ea6703d56243be283d161cb7b79f198318f85
https://github.com/saga-project/BigJob/commit/4f6ea6703d56243be283d161cb7b79f198318f85
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
A examples/hadoop/README
Log Message:
-----------
another README
Commit: 329b5935e402a320c5b2b7c8f0ec8bf3fb30ebd5
https://github.com/saga-project/BigJob/commit/329b5935e402a320c5b2b7c8f0ec8bf3fb30ebd5
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/README.md
Log Message:
-----------
formatting
Commit: f07420ec27be32684c43e7c6bd8c23e839163d1d
https://github.com/saga-project/BigJob/commit/f07420ec27be32684c43e7c6bd8c23e839163d1d
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-01 (Wed, 01 Jan 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/job_plugin/yarn/BigJob-YARN-0.1-SNAPSHOT-jar-with-dependencies.jar
R bigjob/job_plugin/yarn/README.md
M bigjob/job_plugin/yarn/yarn.py
A examples/hadoop/README.md
M examples/hadoop/example-pilot-compute-yarn.py
Log Message:
-----------
enable remote YARN submission
improved README
Commit: be4204f153207c9ffbcb4a13f4b6450c4f9ccf75
https://github.com/saga-project/BigJob/commit/be4204f153207c9ffbcb4a13f4b6450c4f9ccf75
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-01 (Wed, 01 Jan 2014)
Changed paths:
M examples/hadoop/README.md
Log Message:
-----------
README
Commit: 2d3b44ff326af6099cdd560cee9a7c631626301d
https://github.com/saga-project/BigJob/commit/2d3b44ff326af6099cdd560cee9a7c631626301d
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-01 (Wed, 01 Jan 2014)
Changed paths:
R examples/hadoop/README
Log Message:
-----------
removed old README
Commit: 4af38badf197ebdd19e7c5234447330df60bcfd7
https://github.com/saga-project/BigJob/commit/4af38badf197ebdd19e7c5234447330df60bcfd7
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-08 (Wed, 08 Jan 2014)
Changed paths:
M examples/hadoop/README.md
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
logging
Commit: d474b7cbb5486af332ab2df8283f279c41f9809e
https://github.com/saga-project/BigJob/commit/d474b7cbb5486af332ab2df8283f279c41f9809e
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-08 (Wed, 08 Jan 2014)
Changed paths:
M bigjob/bigjob_manager.py
M coordination/bigjob_coordination_redis.py
A tests/performance/redis_timeout.py
Log Message:
-----------
introduced connection pooling across multiple pilots
Commit: 01f51c95e8e570e207451de728fa187c6e8cdf9f
https://github.com/saga-project/BigJob/commit/01f51c95e8e570e207451de728fa187c6e8cdf9f
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-15 (Wed, 15 Jan 2014)
Changed paths:
M pilot/impl/pilot_manager_decentral.py
Log Message:
-----------
removed Melissa's unwanted log output
Commit: 0509c15824f13931721d1f5d496f3927996854d2
https://github.com/saga-project/BigJob/commit/0509c15824f13931721d1f5d496f3927996854d2
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-15 (Wed, 15 Jan 2014)
Changed paths:
M pilot/impl/pilot_manager_decentral.py
Log Message:
-----------
improved parallelization of data movements and cu queuing
Commit: 63e8762fa1176f669480ff1c68a759ef87234a36
https://github.com/saga-project/BigJob/commit/63e8762fa1176f669480ff1c68a759ef87234a36
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-17 (Fri, 17 Jan 2014)
Changed paths:
M pilot/filemanagement/ssh_adaptor.py
Log Message:
-----------
attempt to fix pexpect timeout issue
Commit: 94669ef673856c16f25234c4c45f82056ece4b1e
https://github.com/saga-project/BigJob/commit/94669ef673856c16f25234c4c45f82056ece4b1e
Author: Andre Luckow <andre....@googlemail.com>
Date: 2014-01-17 (Fri, 17 Jan 2014)
Changed paths:
M pilot/filemanagement/ssh_adaptor.py
M setup.py
Log Message:
-----------
another minor fix
Commit: 04607b7ac94b48f923df296d10a726050627ce9b
https://github.com/saga-project/BigJob/commit/04607b7ac94b48f923df296d10a726050627ce9b
Author: Andre Luckow <Andre....@gmail.com>
Date: 2014-01-18 (Sat, 18 Jan 2014)
Changed paths:
A bootstrap/bigjob2-aws-bootstrap.sh
Log Message:
-----------
added AWS bootstrap script for Elastic MapReduce
Commit: e75f99a060f7f9caf5fb09bb6b70039c7558199e
https://github.com/saga-project/BigJob/commit/e75f99a060f7f9caf5fb09bb6b70039c7558199e
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-19 (Sun, 19 Jan 2014)
Changed paths:
M bootstrap/bigjob2-aws-bootstrap.sh
A pilot/filemanagement/hdfs_adaptor.py
M pilot/filemanagement/webhdfs_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
fix bug during file stagin on agent-level
Pilot-Data HDFS adaptor
Commit: a0a69df5019f1cf00caa6566864f617488e962c4
https://github.com/saga-project/BigJob/commit/a0a69df5019f1cf00caa6566864f617488e962c4
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-20 (Mon, 20 Jan 2014)
Changed paths:
M pilot/filemanagement/hdfs_adaptor.py
M pilot/filemanagement/ssh_adaptor.py
R pilot/filemanagement/webhdfs_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
work on HDFS support
Commit: 28206702af2edea6bffee12a33a333ed1e602884
https://github.com/saga-project/BigJob/commit/28206702af2edea6bffee12a33a333ed1e602884
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-20 (Mon, 20 Jan 2014)
Changed paths:
M bigjob/job_plugin/yarn/yarn.py
M bootstrap/bigjob2-aws-bootstrap.sh
Log Message:
-----------
detect YARN and JAVA_HOME
Commit: 29afb6ea7e5267c506d74c62939c0fe86f0acc1d
https://github.com/saga-project/BigJob/commit/29afb6ea7e5267c506d74c62939c0fe86f0acc1d
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-21 (Tue, 21 Jan 2014)
Changed paths:
M bootstrap/bigjob2-aws-bootstrap.sh
Log Message:
-----------
update bootstrap script
Commit: d4236dabc6f35fa7a64638267d8929efdde8ffec
https://github.com/saga-project/BigJob/commit/d4236dabc6f35fa7a64638267d8929efdde8ffec
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-21 (Tue, 21 Jan 2014)
Changed paths:
M bigjob/bigjob_agent.py
M examples/tutorial/barebones-local/local_chained_ensembles.py
M examples/tutorial/barebones-local/local_coupled_ensembles.py
M examples/tutorial/barebones-local/local_simple_ensembles.py
M examples/tutorial/chained_ensembles.py
M examples/tutorial/coupled_ensembles.py
M examples/tutorial/simple_ensemble_dataxfer.py
M examples/tutorial/simple_ensembles.py
M pilot/api/compute/api.py
Log Message:
-----------
Merge branch 'master' into develop
* master:
added safety check
fixed examples so that they do not contain compile errors
minor bug fix in calling the agent via command line
Conflicts:
bigjob/bigjob_agent.py
Commit: ccd30f8d791b88b443471c564265dd5674484298
https://github.com/saga-project/BigJob/commit/ccd30f8d791b88b443471c564265dd5674484298
Author: Andre Merzky <an...@merzky.net>
Date: 2014-01-22 (Wed, 22 Jan 2014)
Changed paths:
M MANIFEST.in
M bigjob.conf
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
A bigjob/job_plugin/mesos/__init__.py
A bigjob/job_plugin/mesos/mesos_executor
A bigjob/job_plugin/mesos/mesos_executor.py
A bigjob/job_plugin/mesos/mesos_job.py
A bigjob/job_plugin/yarn/BigJob-YARN-0.1-SNAPSHOT-jar-with-dependencies.jar
A bigjob/job_plugin/yarn/__init__.py
A bigjob/job_plugin/yarn/yarn.py
M bigjob_dynamic/many_job.py
M bootstrap/bigjob-bootstrap.py
A bootstrap/bigjob2-aws-bootstrap.sh
A bootstrap/bigjob2-bootstrap.py
A bootstrap/bigjob2-bootstrap.sh
M bootstrap/generate_bigjob_bootstrap_script.py
M coordination/bigjob_coordination_redis.py
R distribute_setup.py
M docs/source/conf.py
M docs/source/patterns/chained.rst
R docs/source/usage/appwriting.rst
M docs/source/usage/index.rst
R docs/source/usage/pilotdata.rst
A examples/hadoop/README.md
A examples/hadoop/example-pilot-compute-mesos.py
A examples/hadoop/example-pilot-compute-yarn.py
M examples/tutorial/barebones-local/local_coupled_ensembles.py
M examples/tutorial/barebones-local/local_simple_ensembles.py
M examples/tutorial/coupled_ensembles.py
M examples/tutorial/simple_ensemble_dataxfer.py
M examples/tutorial/simple_ensembles.py
M pilot/api/compute/api.py
A pilot/filemanagement/hdfs_adaptor.py
M pilot/filemanagement/ssh_adaptor.py
R pilot/filemanagement/webhdfs_adaptor.py
M pilot/impl/pilot_manager_decentral.py
M pilot/impl/pilotcompute_manager.py
M pilot/impl/pilotdata_manager.py
M setup.py
A tests/performance/redis_timeout.py
Log Message:
-----------
merge
Commit: 049cf1e650e39d40b99fc5ad4a71bad3243a1837
https://github.com/saga-project/BigJob/commit/049cf1e650e39d40b99fc5ad4a71bad3243a1837
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M bootstrap/bigjob-bootstrap.py
A ez_setup.py
M setup.py
Log Message:
-----------
Merge branch 'master' into develop
* master:
force setuptools 2.x switch
tinkering with setuptools 2.x
use setuptools 2.1 for packaging bj
0.59.1
added VERSION to package
Conflicts:
MANIFEST.in
Commit: 87de769ae5d2ed243f5be59ca7df1ecee7f35d0d
https://github.com/saga-project/BigJob/commit/87de769ae5d2ed243f5be59ca7df1ecee7f35d0d
Author: Pradeep <prade...@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M pilot/impl/pilot_manager_decentral.py
Log Message:
-----------
minor optimization to wait function
Commit: 394d67863b67f16693b8ee5b033160f0b4cef4b0
https://github.com/saga-project/BigJob/commit/394d67863b67f16693b8ee5b033160f0b4cef4b0
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M pilot/impl/pilot_manager_decentral.py
Log Message:
-----------
Merge branch 'develop' of github.com:saga-project/BigJob into develop
Commit: 2b3b9f6d78b3be84f0ab1d54305348d74c763cae
https://github.com/saga-project/BigJob/commit/2b3b9f6d78b3be84f0ab1d54305348d74c763cae
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M bigjob.conf
M bigjob/__init__.py
M bigjob/bigjob_agent.py
M pilot/impl/pilot_manager_decentral.py
M pilot/impl/pilotdata_manager.py
A tests/performance/du_creation.py
Log Message:
-----------
Added threadpooling for data transfers managed by ComputeDataService
Commit: 54a5674c4f5c2ccbbb71971cd84fa957137ce7a3
https://github.com/saga-project/BigJob/commit/54a5674c4f5c2ccbbb71971cd84fa957137ce7a3
Author: Andre Luckow <andre....@googlemail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M bootstrap/bigjob-bootstrap.py
Log Message:
-----------
use existing virtualenv version if possible to avoid setuptools conflict
Commit: cd65bc4ef0d5418ff8741a907d888deab8f6d531
https://github.com/saga-project/BigJob/commit/cd65bc4ef0d5418ff8741a907d888deab8f6d531
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M bootstrap/bigjob2-bootstrap.sh
Log Message:
-----------
update bootstrap script for YARN
Commit: 327b87478d0bc91527bd57c0b62663aa090b3b87
https://github.com/saga-project/BigJob/commit/327b87478d0bc91527bd57c0b62663aa090b3b87
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M bootstrap/bigjob2-bootstrap.py
Log Message:
-----------
update bootstrap for YARN BJ
Commit: 2c833bfe5fd5817d16ef100758af77d526dc6189
https://github.com/saga-project/BigJob/commit/2c833bfe5fd5817d16ef100758af77d526dc6189
Author: Andre Merzky <an...@merzky.net>
Date: 2014-02-14 (Fri, 14 Feb 2014)
Changed paths:
M bigjob.conf
M bigjob/__init__.py
M bigjob/bigjob_agent.py
M bootstrap/bigjob-bootstrap.py
M bootstrap/bigjob2-bootstrap.py
M bootstrap/bigjob2-bootstrap.sh
A ez_setup.py
M pilot/impl/pilot_manager_decentral.py
M pilot/impl/pilotdata_manager.py
M setup.py
A tests/performance/du_creation.py
Log Message:
-----------
Merge branch 'develop' of github.com:saga-project/BigJob into develop
Commit: 7edfadb40ba3ad157bb34dc66c1a4907d602f02b
https://github.com/saga-project/BigJob/commit/7edfadb40ba3ad157bb34dc66c1a4907d602f02b
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-06 (Thu, 06 Mar 2014)
Changed paths:
M bigjob/bigjob_manager.py
M bootstrap/bigjob-bootstrap.py
M coordination/bigjob_coordination_redis.py
Log Message:
-----------
Merge branch 'master' into develop
* master:
Catch file not found exception
use existing virtualenv version if possible to avoid setuptools conflict
Conflicts:
bootstrap/bigjob-bootstrap.py
Commit: f286af9e1e0c480ede9b44c203ddf0b6a56ae77e
https://github.com/saga-project/BigJob/commit/f286af9e1e0c480ede9b44c203ddf0b6a56ae77e
Author: Pradeep <prade...@gmail.com>
Date: 2014-02-09 (Sun, 09 Feb 2014)
Changed paths:
M bigjob/bigjob_agent.py
A examples/pilot-api/example-partial-du-cu-copy.py
M pilot/filemanagement/ssh_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
"Partial du copy to cu enabled"
Commit: 96fe9bc9160219b27fad0d391b520fd271a00937
https://github.com/saga-project/BigJob/commit/96fe9bc9160219b27fad0d391b520fd271a00937
Author: Pradeep <prade...@gmail.com>
Date: 2014-03-06 (Thu, 06 Mar 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
M coordination/bigjob_coordination_redis.py
M pilot/impl/pilotcompute_manager.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
Added method to access the number of nodes by Pilot-Manager, acquired by
a Pilot.
Commit: ba4cd619a12eee90209c72f3bca0adfbb7fcfa81
https://github.com/saga-project/BigJob/commit/ba4cd619a12eee90209c72f3bca0adfbb7fcfa81
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-06 (Thu, 06 Mar 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
M coordination/bigjob_coordination_redis.py
A examples/pilot-api/example-partial-du-cu-copy.py
M pilot/filemanagement/ssh_adaptor.py
M pilot/impl/pilotcompute_manager.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
Merge branch 'Pradeep' into develop
* Pradeep:
Added method to access the number of nodes by Pilot-Manager, acquired by a Pilot.
"Partial du copy to cu enabled"
Commit: e6170a6aa356017533df5874ecda7e996865f64a
https://github.com/saga-project/BigJob/commit/e6170a6aa356017533df5874ecda7e996865f64a
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-16 (Sun, 16 Mar 2014)
Changed paths:
M examples/pilot-api/example-pilot-compute-data.py
A pilot/iterative/__init__.py
A pilot/iterative/iterative_redis.py
A pilot/iterative/iterative_tachyon.py
A pilot/iterative/rediscluster
Log Message:
-----------
benchmarking backends for iterative Pilot-Data
Commit: 997e2c15f88bf56da71286df8b9c69de15bb50c6
https://github.com/saga-project/BigJob/commit/997e2c15f88bf56da71286df8b9c69de15bb50c6
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-22 (Sat, 22 Mar 2014)
Changed paths:
M pilot/iterative/iterative_redis.py
M pilot/iterative/iterative_tachyon.py
Log Message:
-----------
refinements to tests
Commit: e49de062bb5b766f58254dd235cd41c84b3e94cc
https://github.com/saga-project/BigJob/commit/e49de062bb5b766f58254dd235cd41c84b3e94cc
Author: Andre <andre....@gmail.com>
Date: 2014-03-22 (Sat, 22 Mar 2014)
Changed paths:
M pilot/iterative/iterative_tachyon.py
Log Message:
-----------
use env variable
Commit: 618496861ae8e94c66cecbc0dfb841edf42ac467
https://github.com/saga-project/BigJob/commit/618496861ae8e94c66cecbc0dfb841edf42ac467
Author: Andre <andre....@gmail.com>
Date: 2014-03-23 (Sun, 23 Mar 2014)
Changed paths:
M pilot/iterative/iterative_tachyon.py
Log Message:
-----------
tunign
Commit: ea59bed9fdae0781410529ac798c9ceb3a3f7b67
https://github.com/saga-project/BigJob/commit/ea59bed9fdae0781410529ac798c9ceb3a3f7b67
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-22 (Sat, 22 Mar 2014)
Changed paths:
A pilot/iterative/README.md
A pilot/iterative/log.txt
R pilot/iterative/rediscluster
A pilot/iterative/rediscluster/rediscluster.py
Log Message:
-----------
rediscluster
Commit: 51c706d8c8410e1b087817ce4d38f1555a842975
https://github.com/saga-project/BigJob/commit/51c706d8c8410e1b087817ce4d38f1555a842975
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-22 (Sat, 22 Mar 2014)
Changed paths:
A pilot/iterative/rediscluster/__init__.py
Log Message:
-----------
module file
Commit: 74a140115635f42a71c0f1c0083dec32506a1aea
https://github.com/saga-project/BigJob/commit/74a140115635f42a71c0f1c0083dec32506a1aea
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-06-22 (Sun, 22 Jun 2014)
Changed paths:
A pilot/iterative/iterative_hdfs.py
Log Message:
-----------
HDFS caching abstraction
Commit: d5dbeb4e0fa3166758aced744249e89be71a2891
https://github.com/saga-project/BigJob/commit/d5dbeb4e0fa3166758aced744249e89be71a2891
Author: Andre <andre....@gmail.com>
Date: 2014-03-23 (Sun, 23 Mar 2014)
Changed paths:
A pilot/iterative/clean.sh
M pilot/iterative/iterative_redis.py
M pilot/iterative/iterative_tachyon.py
M setup.py
Log Message:
-----------
further refinements
Commit: 62c1f0702191e6ce29011ab3447a6d39eb159953
https://github.com/saga-project/BigJob/commit/62c1f0702191e6ce29011ab3447a6d39eb159953
Author: Andre <andre....@gmail.com>
Date: 2014-03-23 (Sun, 23 Mar 2014)
Changed paths:
M pilot/iterative/iterative_redis.py
M pilot/iterative/iterative_tachyon.py
Log Message:
-----------
updates for scale-out test
Commit: f6d54f8a89c43abb0443f4439346b1d2e0116798
https://github.com/saga-project/BigJob/commit/f6d54f8a89c43abb0443f4439346b1d2e0116798
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-30 (Sun, 30 Mar 2014)
Changed paths:
M examples/hadoop/README.md
Log Message:
-----------
Update README.md
added YARN config
Commit: 70beebce2c6f63abd4b86d3f73fae416628aa3ab
https://github.com/saga-project/BigJob/commit/70beebce2c6f63abd4b86d3f73fae416628aa3ab
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-06-22 (Sun, 22 Jun 2014)
Changed paths:
M examples/hadoop/README.md
A pilot/iterative/clean.sh
M pilot/iterative/iterative_redis.py
M pilot/iterative/iterative_tachyon.py
M setup.py
Log Message:
-----------
Merge branch 'develop' of github.com:saga-project/BigJob into develop
Commit: 7d315a4acb6aab19cc3a752b763927263ae7bd1a
https://github.com/saga-project/BigJob/commit/7d315a4acb6aab19cc3a752b763927263ae7bd1a
Author: Andre <andre....@gmail.com>
Date: 2014-06-22 (Sun, 22 Jun 2014)
Changed paths:
M pilot/filemanagement/hdfs_adaptor.py
M pilot/iterative/iterative_hdfs.py
M pilot/iterative/iterative_redis.py
M pilot/iterative/iterative_tachyon.py
Log Message:
-----------
further tests
Commit: 666bdd2419bdb8776d5f3dc1ba8da110117637f9
https://github.com/saga-project/BigJob/commit/666bdd2419bdb8776d5f3dc1ba8da110117637f9
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-06-22 (Sun, 22 Jun 2014)
Changed paths:
M pilot/iterative/iterative_hdfs.py
Log Message:
-----------
added caching test
Commit: 88969f02ace2d717b89dac31bb10f4b985a09f2a
https://github.com/saga-project/BigJob/commit/88969f02ace2d717b89dac31bb10f4b985a09f2a
Author: Andre Merzky <an...@merzky.net>
Date: 2014-07-13 (Sun, 13 Jul 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
M bootstrap/bigjob-bootstrap.py
M coordination/bigjob_coordination_redis.py
M examples/hadoop/README.md
A examples/pilot-api/example-partial-du-cu-copy.py
M examples/pilot-api/example-pilot-compute-data.py
M pilot/filemanagement/hdfs_adaptor.py
M pilot/filemanagement/ssh_adaptor.py
M pilot/impl/pilotcompute_manager.py
M pilot/impl/pilotdata_manager.py
A pilot/iterative/README.md
A pilot/iterative/__init__.py
A pilot/iterative/clean.sh
A pilot/iterative/iterative_hdfs.py
A pilot/iterative/iterative_redis.py
A pilot/iterative/iterative_tachyon.py
A pilot/iterative/log.txt
A pilot/iterative/rediscluster/__init__.py
A pilot/iterative/rediscluster/rediscluster.py
M setup.py
Log Message:
-----------
Merge branch 'develop' of github.com:saga-project/BigJob into develop
Commit: 852f84e71419890bedf77a7f1f3edc4e678bf2e0
https://github.com/saga-project/BigJob/commit/852f84e71419890bedf77a7f1f3edc4e678bf2e0
Author: Pradeep Kumar Mantha <pma...@login2.stampede.tacc.utexas.edu>
Date: 2014-07-27 (Sun, 27 Jul 2014)
Changed paths:
M bigjob/bigjob_manager.py
M bootstrap/bigjob-bootstrap.py
M examples/pilot-api/example-pilot-api.py
Log Message:
-----------
bootstrap BJ2
Commit: 832db25912701880702db75a0167593e665c1446
https://github.com/saga-project/BigJob/commit/832db25912701880702db75a0167593e665c1446
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-08-31 (Sun, 31 Aug 2014)
Changed paths:
R api/__init__.py
R api/base.py
M bigjob/__init__.py
M bigjob/bigjob_manager.py
M setup.py
Log Message:
-----------
some refactoring
Commit: 291eb34eef478072287dd9675907edb207050110
https://github.com/saga-project/BigJob/commit/291eb34eef478072287dd9675907edb207050110
Author: pmantha <pma...@salesforce.com>
Date: 2014-10-19 (Sun, 19 Oct 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
Log Message:
-----------
get nodes fix
Commit: 2da7592ea610725d214f8e7e46d7a69f016f81c5
https://github.com/saga-project/BigJob/commit/2da7592ea610725d214f8e7e46d7a69f016f81c5
Author: pradeepmantha <prade...@gmail.com>
Date: 2014-10-20 (Mon, 20 Oct 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
Log Message:
-----------
Merge pull request #179 from saga-project/get_nodes
get nodes fix
Commit: 9c37d8c4e4790f58346cb8330604e3821235e481
https://github.com/saga-project/BigJob/commit/9c37d8c4e4790f58346cb8330604e3821235e481
Author: pmantha <pma...@salesforce.com>
Date: 2014-10-27 (Mon, 27 Oct 2014)
Changed paths:
M bigjob/__init__.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
modified DU add_files to register files in redis, which are already in DU
Commit: 8fc903782ff346c37804f7df92b5bc3cfde37365
https://github.com/saga-project/BigJob/commit/8fc903782ff346c37804f7df92b5bc3cfde37365
Author: Pradeep <prade...@gmail.com>
Date: 2014-10-29 (Wed, 29 Oct 2014)
Changed paths:
M pilot/coordination/redis_adaptor.py
M setup.py
Log Message:
-----------
too many connections ...
Commit: 767e5a9a637e4f85b41b98f47f8fde4ac4e15319
https://github.com/saga-project/BigJob/commit/767e5a9a637e4f85b41b98f47f8fde4ac4e15319
Author: pmantha <pma...@salesforce.com>
Date: 2014-11-01 (Sat, 01 Nov 2014)
Changed paths:
M bigjob/bigjob_agent.py
M pilot/coordination/redis_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
input du url restoration for each sj is costly..
Commit: 1e3ecb3a8a4e41b0bb55b31615566530b14197c4
https://github.com/saga-project/BigJob/commit/1e3ecb3a8a4e41b0bb55b31615566530b14197c4
Author: pmantha <pma...@salesforce.com>
Date: 2014-11-01 (Sat, 01 Nov 2014)
Changed paths:
M bigjob/bigjob_agent.py
Log Message:
-----------
minor fix
Compare: https://github.com/saga-project/BigJob/compare/1364772d79bb...1e3ecb3a8a4e
To unsubscribe from these emails, change your notification settings at https://github.com/saga-project/BigJob/settings/notifications
Home: https://github.com/saga-project/BigJob
Commit: a9ac32f82abc70df2c3180328785e2dde417c0ab
https://github.com/saga-project/BigJob/commit/a9ac32f82abc70df2c3180328785e2dde417c0ab
Author: Andre Merzky <an...@merzky.net>
Date: 2014-01-06 (Mon, 06 Jan 2014)
Changed paths:
M examples/tutorial/simple_ensembles.py
Log Message:
-----------
examples should not have syntax errors, in particular if they are installed...
Commit: c86764b62909e3290a24054af87b0017784e92ed
https://github.com/saga-project/BigJob/commit/c86764b62909e3290a24054af87b0017784e92ed
Author: Andre Merzky <an...@merzky.net>
Date: 2014-01-06 (Mon, 06 Jan 2014)
Changed paths:
M examples/tutorial/barebones-local/local_chained_ensembles.py
M examples/tutorial/barebones-local/local_coupled_ensembles.py
M examples/tutorial/barebones-local/local_simple_ensembles.py
M examples/tutorial/chained_ensembles.py
M examples/tutorial/coupled_ensembles.py
M examples/tutorial/simple_ensemble_dataxfer.py
M examples/tutorial/simple_ensembles.py
Log Message:
-----------
examples should not have syntax errors, in particular if they are installed... And tabs are bad, too!
Commit: 4b41cc6ea7c97caa8fcede29f19c94c0be80c7c7
https://github.com/saga-project/BigJob/commit/4b41cc6ea7c97caa8fcede29f19c94c0be80c7c7
Author: Mark Santcroos <mas...@scarletmail.rutgers.edu>
Date: 2013-12-05 (Thu, 05 Dec 2013)
Changed paths:
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
Report the service_url in the exception to assist debugging.
Commit: adea1098cca668e864aa2722661898ab61b100eb
https://github.com/saga-project/BigJob/commit/adea1098cca668e864aa2722661898ab61b100eb
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-05 (Thu, 05 Dec 2013)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob_dynamic/many_job.py
M bootstrap/bigjob-bootstrap.py
M docs/source/conf.py
M docs/source/patterns/chained.rst
R docs/source/usage/appwriting.rst
M docs/source/usage/index.rst
R docs/source/usage/pilotdata.rst
M examples/tutorial/simple_ensembles.py
M pilot/impl/pilotcompute_manager.py
Log Message:
-----------
Merge branch 'master' into develop
# By Melissa Romanus (6) and Andre Luckow (2)
# Via Melissa Romanus
* master:
Fix indices
Remove appwriting and pilotdata, these should not be front-facing pages anymore
upgraded BJ to latest virtualenv/distribute
tabs vs. spaces
Using explicit spaces
PCD needs to take a None string for new more expressive examples to work
BigJob moves to cleaner theme. Will no longer match saga-python
Fix issue 158 for mturilli
Commit: 69235c30dcb80d3b82a95c41e3129217856dfe51
https://github.com/saga-project/BigJob/commit/69235c30dcb80d3b82a95c41e3129217856dfe51
Author: Mark Santcroos <mas...@scarletmail.rutgers.edu>
Date: 2013-12-06 (Fri, 06 Dec 2013)
Changed paths:
M bigjob/bigjob_manager.py
Log Message:
-----------
Convert and correct spurious print into a debug message.
Commit: fadc00a445a5474be795c3276cd1417f0ff515d2
https://github.com/saga-project/BigJob/commit/fadc00a445a5474be795c3276cd1417f0ff515d2
Author: Mark Santcroos <mas...@scarletmail.rutgers.edu>
Date: 2013-12-06 (Fri, 06 Dec 2013)
Changed paths:
M pilot/filemanagement/ssh_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
Create parent dirs also, for both ssh and local.
Commit: a3613787d81d1d7954f4daa9ddd9bbf97dcb971f
https://github.com/saga-project/BigJob/commit/a3613787d81d1d7954f4daa9ddd9bbf97dcb971f
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-18 (Wed, 18 Dec 2013)
Changed paths:
M bigjob/bigjob_manager.py
M pilot/filemanagement/ssh_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
Merge branch 'develop' of github.com:saga-project/BigJob into develop
Commit: 473f8fed78bd9ec3845fe5f56d0a74b8e4a43ef8
https://github.com/saga-project/BigJob/commit/473f8fed78bd9ec3845fe5f56d0a74b8e4a43ef8
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-18 (Wed, 18 Dec 2013)
Changed paths:
M bigjob.conf
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
A bigjob/job_plugin/mesos/README
A bigjob/job_plugin/mesos/__init__.py
A bigjob/job_plugin/mesos/mesos_executor
A bigjob/job_plugin/mesos/mesos_executor.py
A bigjob/job_plugin/mesos/mesos_job.py
R distribute_setup.py
A examples/bigdata/example-pilot-compute-mesos.py
M setup.py
Log Message:
-----------
Initial support for Mesos scheduler
Commit: f5c695fe8c441d64ab96cce47bbf3a4c24f05bbe
https://github.com/saga-project/BigJob/commit/f5c695fe8c441d64ab96cce47bbf3a4c24f05bbe
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
R bigjob/job_plugin/mesos/README
M bigjob/job_plugin/mesos/mesos_job.py
A bigjob/job_plugin/yarn/README.md
A bigjob/job_plugin/yarn/__init__.py
A bigjob/job_plugin/yarn/yarn.py
A bootstrap/bigjob-bootstrap.sh
A examples/bigdata/example-pilot-compute-yarn.py
Log Message:
-----------
Work on YARN plugin
Commit: 735d57dfe054290276e7fcc3c674f40654fba78b
https://github.com/saga-project/BigJob/commit/735d57dfe054290276e7fcc3c674f40654fba78b
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/job_plugin/yarn/yarn.py
R bootstrap/bigjob-bootstrap.sh
A bootstrap/bigjob2-bootstrap.py
A bootstrap/bigjob2-bootstrap.sh
M setup.py
Log Message:
-----------
added new PyPi version to test new Bootstrapping method
Commit: 2d1153c4753aeb9362a27403ad21768431479be2
https://github.com/saga-project/BigJob/commit/2d1153c4753aeb9362a27403ad21768431479be2
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/yarn.py
M bootstrap/bigjob2-bootstrap.sh
Log Message:
-----------
minor tweaks to yarn
Commit: 8d8aa283ab5351333de6a4f4639d5874b8643590
https://github.com/saga-project/BigJob/commit/8d8aa283ab5351333de6a4f4639d5874b8643590
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M MANIFEST.in
Log Message:
-----------
update MANIFESt
Commit: c4150133fa2de9f0bef061b775d6ce35e287cf38
https://github.com/saga-project/BigJob/commit/c4150133fa2de9f0bef061b775d6ce35e287cf38
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M bootstrap/bigjob2-bootstrap.py
M bootstrap/generate_bigjob_bootstrap_script.py
Log Message:
-----------
refined bigjob2 bootstrap
Commit: 1f1633bdc0cc20da65b4c3142ef06df05450fcf9
https://github.com/saga-project/BigJob/commit/1f1633bdc0cc20da65b4c3142ef06df05450fcf9
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-21 (Sat, 21 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/yarn.py
M bootstrap/bigjob2-bootstrap.py
Log Message:
-----------
fixed bootstrap for YARN
Commit: 6ca38f7c2fd4f346844b34b743c561136f9acbeb
https://github.com/saga-project/BigJob/commit/6ca38f7c2fd4f346844b34b743c561136f9acbeb
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
M bigjob/job_plugin/mesos/mesos_job.py
M bigjob/job_plugin/yarn/yarn.py
M bootstrap/bigjob2-bootstrap.sh
R examples/bigdata/example-pilot-compute-mesos.py
R examples/bigdata/example-pilot-compute-yarn.py
Log Message:
-----------
further work on YARN adaptor and bootstrapping
Commit: 8fdbd7b0b03db2776d05bba4589e2d4e557a4f43
https://github.com/saga-project/BigJob/commit/8fdbd7b0b03db2776d05bba4589e2d4e557a4f43
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
A examples/hadoop/example-pilot-compute-mesos.py
A examples/hadoop/example-pilot-compute-yarn.py
Log Message:
-----------
further work on YARN adaptor and bootstrapping
Commit: 0294d27694187fa51675497997385a562de41ecd
https://github.com/saga-project/BigJob/commit/0294d27694187fa51675497997385a562de41ecd
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/yarn.py
Log Message:
-----------
removed debugging output
Commit: 913d633b7a8b848ce4ccd43000ab8786a5e7ca2e
https://github.com/saga-project/BigJob/commit/913d633b7a8b848ce4ccd43000ab8786a5e7ca2e
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
A bigjob/job_plugin/yarn/BigJob-YARN-0.1-SNAPSHOT-jar-with-dependencies.jar
M bigjob/job_plugin/yarn/yarn.py
Log Message:
-----------
for simplicity added YARN BigJob jar
Commit: dbfe26c753e55d54674c58cf238c2fd399596e87
https://github.com/saga-project/BigJob/commit/dbfe26c753e55d54674c58cf238c2fd399596e87
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/yarn.py
M examples/hadoop/example-pilot-compute-yarn.py
Log Message:
-----------
fixes
Commit: 375031788c44edb6726b20f91afb69e2f41e745d
https://github.com/saga-project/BigJob/commit/375031788c44edb6726b20f91afb69e2f41e745d
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/README.md
Log Message:
-----------
readme expanded
Commit: 4f6ea6703d56243be283d161cb7b79f198318f85
https://github.com/saga-project/BigJob/commit/4f6ea6703d56243be283d161cb7b79f198318f85
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
A examples/hadoop/README
Log Message:
-----------
another README
Commit: 329b5935e402a320c5b2b7c8f0ec8bf3fb30ebd5
https://github.com/saga-project/BigJob/commit/329b5935e402a320c5b2b7c8f0ec8bf3fb30ebd5
Author: Andre Luckow <andre....@gmail.com>
Date: 2013-12-23 (Mon, 23 Dec 2013)
Changed paths:
M bigjob/job_plugin/yarn/README.md
Log Message:
-----------
formatting
Commit: f07420ec27be32684c43e7c6bd8c23e839163d1d
https://github.com/saga-project/BigJob/commit/f07420ec27be32684c43e7c6bd8c23e839163d1d
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-01 (Wed, 01 Jan 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/job_plugin/yarn/BigJob-YARN-0.1-SNAPSHOT-jar-with-dependencies.jar
R bigjob/job_plugin/yarn/README.md
M bigjob/job_plugin/yarn/yarn.py
A examples/hadoop/README.md
M examples/hadoop/example-pilot-compute-yarn.py
Log Message:
-----------
enable remote YARN submission
improved README
Commit: be4204f153207c9ffbcb4a13f4b6450c4f9ccf75
https://github.com/saga-project/BigJob/commit/be4204f153207c9ffbcb4a13f4b6450c4f9ccf75
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-01 (Wed, 01 Jan 2014)
Changed paths:
M examples/hadoop/README.md
Log Message:
-----------
README
Commit: 2d3b44ff326af6099cdd560cee9a7c631626301d
https://github.com/saga-project/BigJob/commit/2d3b44ff326af6099cdd560cee9a7c631626301d
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-01 (Wed, 01 Jan 2014)
Changed paths:
R examples/hadoop/README
Log Message:
-----------
removed old README
Commit: 4af38badf197ebdd19e7c5234447330df60bcfd7
https://github.com/saga-project/BigJob/commit/4af38badf197ebdd19e7c5234447330df60bcfd7
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-08 (Wed, 08 Jan 2014)
Changed paths:
M examples/hadoop/README.md
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
logging
Commit: d474b7cbb5486af332ab2df8283f279c41f9809e
https://github.com/saga-project/BigJob/commit/d474b7cbb5486af332ab2df8283f279c41f9809e
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-08 (Wed, 08 Jan 2014)
Changed paths:
M bigjob/bigjob_manager.py
M coordination/bigjob_coordination_redis.py
A tests/performance/redis_timeout.py
Log Message:
-----------
introduced connection pooling across multiple pilots
Commit: 01f51c95e8e570e207451de728fa187c6e8cdf9f
https://github.com/saga-project/BigJob/commit/01f51c95e8e570e207451de728fa187c6e8cdf9f
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-15 (Wed, 15 Jan 2014)
Changed paths:
M pilot/impl/pilot_manager_decentral.py
Log Message:
-----------
removed Melissa's unwanted log output
Commit: 0509c15824f13931721d1f5d496f3927996854d2
https://github.com/saga-project/BigJob/commit/0509c15824f13931721d1f5d496f3927996854d2
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-15 (Wed, 15 Jan 2014)
Changed paths:
M pilot/impl/pilot_manager_decentral.py
Log Message:
-----------
improved parallelization of data movements and cu queuing
Commit: 63e8762fa1176f669480ff1c68a759ef87234a36
https://github.com/saga-project/BigJob/commit/63e8762fa1176f669480ff1c68a759ef87234a36
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-17 (Fri, 17 Jan 2014)
Changed paths:
M pilot/filemanagement/ssh_adaptor.py
Log Message:
-----------
attempt to fix pexpect timeout issue
Commit: 94669ef673856c16f25234c4c45f82056ece4b1e
https://github.com/saga-project/BigJob/commit/94669ef673856c16f25234c4c45f82056ece4b1e
Author: Andre Luckow <andre....@googlemail.com>
Date: 2014-01-17 (Fri, 17 Jan 2014)
Changed paths:
M pilot/filemanagement/ssh_adaptor.py
M setup.py
Log Message:
-----------
another minor fix
Commit: 04607b7ac94b48f923df296d10a726050627ce9b
https://github.com/saga-project/BigJob/commit/04607b7ac94b48f923df296d10a726050627ce9b
Author: Andre Luckow <Andre....@gmail.com>
Date: 2014-01-18 (Sat, 18 Jan 2014)
Changed paths:
A bootstrap/bigjob2-aws-bootstrap.sh
Log Message:
-----------
added AWS bootstrap script for Elastic MapReduce
Commit: e75f99a060f7f9caf5fb09bb6b70039c7558199e
https://github.com/saga-project/BigJob/commit/e75f99a060f7f9caf5fb09bb6b70039c7558199e
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-19 (Sun, 19 Jan 2014)
Changed paths:
M bootstrap/bigjob2-aws-bootstrap.sh
A pilot/filemanagement/hdfs_adaptor.py
M pilot/filemanagement/webhdfs_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
fix bug during file stagin on agent-level
Pilot-Data HDFS adaptor
Commit: a0a69df5019f1cf00caa6566864f617488e962c4
https://github.com/saga-project/BigJob/commit/a0a69df5019f1cf00caa6566864f617488e962c4
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-20 (Mon, 20 Jan 2014)
Changed paths:
M pilot/filemanagement/hdfs_adaptor.py
M pilot/filemanagement/ssh_adaptor.py
R pilot/filemanagement/webhdfs_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
work on HDFS support
Commit: 28206702af2edea6bffee12a33a333ed1e602884
https://github.com/saga-project/BigJob/commit/28206702af2edea6bffee12a33a333ed1e602884
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-20 (Mon, 20 Jan 2014)
Changed paths:
M bigjob/job_plugin/yarn/yarn.py
M bootstrap/bigjob2-aws-bootstrap.sh
Log Message:
-----------
detect YARN and JAVA_HOME
Commit: 29afb6ea7e5267c506d74c62939c0fe86f0acc1d
https://github.com/saga-project/BigJob/commit/29afb6ea7e5267c506d74c62939c0fe86f0acc1d
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-21 (Tue, 21 Jan 2014)
Changed paths:
M bootstrap/bigjob2-aws-bootstrap.sh
Log Message:
-----------
update bootstrap script
Commit: d4236dabc6f35fa7a64638267d8929efdde8ffec
https://github.com/saga-project/BigJob/commit/d4236dabc6f35fa7a64638267d8929efdde8ffec
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-21 (Tue, 21 Jan 2014)
Changed paths:
M bigjob/bigjob_agent.py
M examples/tutorial/barebones-local/local_chained_ensembles.py
M examples/tutorial/barebones-local/local_coupled_ensembles.py
M examples/tutorial/barebones-local/local_simple_ensembles.py
M examples/tutorial/chained_ensembles.py
M examples/tutorial/coupled_ensembles.py
M examples/tutorial/simple_ensemble_dataxfer.py
M examples/tutorial/simple_ensembles.py
M pilot/api/compute/api.py
Log Message:
-----------
Merge branch 'master' into develop
* master:
added safety check
fixed examples so that they do not contain compile errors
minor bug fix in calling the agent via command line
Conflicts:
bigjob/bigjob_agent.py
Commit: ccd30f8d791b88b443471c564265dd5674484298
https://github.com/saga-project/BigJob/commit/ccd30f8d791b88b443471c564265dd5674484298
Author: Andre Merzky <an...@merzky.net>
Date: 2014-01-22 (Wed, 22 Jan 2014)
Changed paths:
M MANIFEST.in
M bigjob.conf
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
A bigjob/job_plugin/mesos/__init__.py
A bigjob/job_plugin/mesos/mesos_executor
A bigjob/job_plugin/mesos/mesos_executor.py
A bigjob/job_plugin/mesos/mesos_job.py
A bigjob/job_plugin/yarn/BigJob-YARN-0.1-SNAPSHOT-jar-with-dependencies.jar
A bigjob/job_plugin/yarn/__init__.py
A bigjob/job_plugin/yarn/yarn.py
M bigjob_dynamic/many_job.py
M bootstrap/bigjob-bootstrap.py
A bootstrap/bigjob2-aws-bootstrap.sh
A bootstrap/bigjob2-bootstrap.py
A bootstrap/bigjob2-bootstrap.sh
M bootstrap/generate_bigjob_bootstrap_script.py
M coordination/bigjob_coordination_redis.py
R distribute_setup.py
M docs/source/conf.py
M docs/source/patterns/chained.rst
R docs/source/usage/appwriting.rst
M docs/source/usage/index.rst
R docs/source/usage/pilotdata.rst
A examples/hadoop/README.md
A examples/hadoop/example-pilot-compute-mesos.py
A examples/hadoop/example-pilot-compute-yarn.py
M examples/tutorial/barebones-local/local_coupled_ensembles.py
M examples/tutorial/barebones-local/local_simple_ensembles.py
M examples/tutorial/coupled_ensembles.py
M examples/tutorial/simple_ensemble_dataxfer.py
M examples/tutorial/simple_ensembles.py
M pilot/api/compute/api.py
A pilot/filemanagement/hdfs_adaptor.py
M pilot/filemanagement/ssh_adaptor.py
R pilot/filemanagement/webhdfs_adaptor.py
M pilot/impl/pilot_manager_decentral.py
M pilot/impl/pilotcompute_manager.py
M pilot/impl/pilotdata_manager.py
M setup.py
A tests/performance/redis_timeout.py
Log Message:
-----------
merge
Commit: 049cf1e650e39d40b99fc5ad4a71bad3243a1837
https://github.com/saga-project/BigJob/commit/049cf1e650e39d40b99fc5ad4a71bad3243a1837
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M bootstrap/bigjob-bootstrap.py
A ez_setup.py
M setup.py
Log Message:
-----------
Merge branch 'master' into develop
* master:
force setuptools 2.x switch
tinkering with setuptools 2.x
use setuptools 2.1 for packaging bj
0.59.1
added VERSION to package
Conflicts:
MANIFEST.in
Commit: 87de769ae5d2ed243f5be59ca7df1ecee7f35d0d
https://github.com/saga-project/BigJob/commit/87de769ae5d2ed243f5be59ca7df1ecee7f35d0d
Author: Pradeep <prade...@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M pilot/impl/pilot_manager_decentral.py
Log Message:
-----------
minor optimization to wait function
Commit: 394d67863b67f16693b8ee5b033160f0b4cef4b0
https://github.com/saga-project/BigJob/commit/394d67863b67f16693b8ee5b033160f0b4cef4b0
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M pilot/impl/pilot_manager_decentral.py
Log Message:
-----------
Merge branch 'develop' of github.com:saga-project/BigJob into develop
Commit: 2b3b9f6d78b3be84f0ab1d54305348d74c763cae
https://github.com/saga-project/BigJob/commit/2b3b9f6d78b3be84f0ab1d54305348d74c763cae
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M bigjob.conf
M bigjob/__init__.py
M bigjob/bigjob_agent.py
M pilot/impl/pilot_manager_decentral.py
M pilot/impl/pilotdata_manager.py
A tests/performance/du_creation.py
Log Message:
-----------
Added threadpooling for data transfers managed by ComputeDataService
Commit: 54a5674c4f5c2ccbbb71971cd84fa957137ce7a3
https://github.com/saga-project/BigJob/commit/54a5674c4f5c2ccbbb71971cd84fa957137ce7a3
Author: Andre Luckow <andre....@googlemail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M bootstrap/bigjob-bootstrap.py
Log Message:
-----------
use existing virtualenv version if possible to avoid setuptools conflict
Commit: cd65bc4ef0d5418ff8741a907d888deab8f6d531
https://github.com/saga-project/BigJob/commit/cd65bc4ef0d5418ff8741a907d888deab8f6d531
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M bootstrap/bigjob2-bootstrap.sh
Log Message:
-----------
update bootstrap script for YARN
Commit: 327b87478d0bc91527bd57c0b62663aa090b3b87
https://github.com/saga-project/BigJob/commit/327b87478d0bc91527bd57c0b62663aa090b3b87
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-01-25 (Sat, 25 Jan 2014)
Changed paths:
M bootstrap/bigjob2-bootstrap.py
Log Message:
-----------
update bootstrap for YARN BJ
Commit: 2c833bfe5fd5817d16ef100758af77d526dc6189
https://github.com/saga-project/BigJob/commit/2c833bfe5fd5817d16ef100758af77d526dc6189
Author: Andre Merzky <an...@merzky.net>
Date: 2014-02-14 (Fri, 14 Feb 2014)
Changed paths:
M bigjob.conf
M bigjob/__init__.py
M bigjob/bigjob_agent.py
M bootstrap/bigjob-bootstrap.py
M bootstrap/bigjob2-bootstrap.py
M bootstrap/bigjob2-bootstrap.sh
A ez_setup.py
M pilot/impl/pilot_manager_decentral.py
M pilot/impl/pilotdata_manager.py
M setup.py
A tests/performance/du_creation.py
Log Message:
-----------
Merge branch 'develop' of github.com:saga-project/BigJob into develop
Commit: 7edfadb40ba3ad157bb34dc66c1a4907d602f02b
https://github.com/saga-project/BigJob/commit/7edfadb40ba3ad157bb34dc66c1a4907d602f02b
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-06 (Thu, 06 Mar 2014)
Changed paths:
M bigjob/bigjob_manager.py
M bootstrap/bigjob-bootstrap.py
M coordination/bigjob_coordination_redis.py
Log Message:
-----------
Merge branch 'master' into develop
* master:
Catch file not found exception
use existing virtualenv version if possible to avoid setuptools conflict
Conflicts:
bootstrap/bigjob-bootstrap.py
Commit: f286af9e1e0c480ede9b44c203ddf0b6a56ae77e
https://github.com/saga-project/BigJob/commit/f286af9e1e0c480ede9b44c203ddf0b6a56ae77e
Author: Pradeep <prade...@gmail.com>
Date: 2014-02-09 (Sun, 09 Feb 2014)
Changed paths:
M bigjob/bigjob_agent.py
A examples/pilot-api/example-partial-du-cu-copy.py
M pilot/filemanagement/ssh_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
"Partial du copy to cu enabled"
Commit: 96fe9bc9160219b27fad0d391b520fd271a00937
https://github.com/saga-project/BigJob/commit/96fe9bc9160219b27fad0d391b520fd271a00937
Author: Pradeep <prade...@gmail.com>
Date: 2014-03-06 (Thu, 06 Mar 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
M coordination/bigjob_coordination_redis.py
M pilot/impl/pilotcompute_manager.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
Added method to access the number of nodes by Pilot-Manager, acquired by
a Pilot.
Commit: ba4cd619a12eee90209c72f3bca0adfbb7fcfa81
https://github.com/saga-project/BigJob/commit/ba4cd619a12eee90209c72f3bca0adfbb7fcfa81
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-06 (Thu, 06 Mar 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
M coordination/bigjob_coordination_redis.py
A examples/pilot-api/example-partial-du-cu-copy.py
M pilot/filemanagement/ssh_adaptor.py
M pilot/impl/pilotcompute_manager.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
Merge branch 'Pradeep' into develop
* Pradeep:
Added method to access the number of nodes by Pilot-Manager, acquired by a Pilot.
"Partial du copy to cu enabled"
Commit: e6170a6aa356017533df5874ecda7e996865f64a
https://github.com/saga-project/BigJob/commit/e6170a6aa356017533df5874ecda7e996865f64a
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-16 (Sun, 16 Mar 2014)
Changed paths:
M examples/pilot-api/example-pilot-compute-data.py
A pilot/iterative/__init__.py
A pilot/iterative/iterative_redis.py
A pilot/iterative/iterative_tachyon.py
A pilot/iterative/rediscluster
Log Message:
-----------
benchmarking backends for iterative Pilot-Data
Commit: 997e2c15f88bf56da71286df8b9c69de15bb50c6
https://github.com/saga-project/BigJob/commit/997e2c15f88bf56da71286df8b9c69de15bb50c6
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-22 (Sat, 22 Mar 2014)
Changed paths:
M pilot/iterative/iterative_redis.py
M pilot/iterative/iterative_tachyon.py
Log Message:
-----------
refinements to tests
Commit: e49de062bb5b766f58254dd235cd41c84b3e94cc
https://github.com/saga-project/BigJob/commit/e49de062bb5b766f58254dd235cd41c84b3e94cc
Author: Andre <andre....@gmail.com>
Date: 2014-03-22 (Sat, 22 Mar 2014)
Changed paths:
M pilot/iterative/iterative_tachyon.py
Log Message:
-----------
use env variable
Commit: 618496861ae8e94c66cecbc0dfb841edf42ac467
https://github.com/saga-project/BigJob/commit/618496861ae8e94c66cecbc0dfb841edf42ac467
Author: Andre <andre....@gmail.com>
Date: 2014-03-23 (Sun, 23 Mar 2014)
Changed paths:
M pilot/iterative/iterative_tachyon.py
Log Message:
-----------
tunign
Commit: ea59bed9fdae0781410529ac798c9ceb3a3f7b67
https://github.com/saga-project/BigJob/commit/ea59bed9fdae0781410529ac798c9ceb3a3f7b67
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-22 (Sat, 22 Mar 2014)
Changed paths:
A pilot/iterative/README.md
A pilot/iterative/log.txt
R pilot/iterative/rediscluster
A pilot/iterative/rediscluster/rediscluster.py
Log Message:
-----------
rediscluster
Commit: 51c706d8c8410e1b087817ce4d38f1555a842975
https://github.com/saga-project/BigJob/commit/51c706d8c8410e1b087817ce4d38f1555a842975
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-22 (Sat, 22 Mar 2014)
Changed paths:
A pilot/iterative/rediscluster/__init__.py
Log Message:
-----------
module file
Commit: 74a140115635f42a71c0f1c0083dec32506a1aea
https://github.com/saga-project/BigJob/commit/74a140115635f42a71c0f1c0083dec32506a1aea
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-06-22 (Sun, 22 Jun 2014)
Changed paths:
A pilot/iterative/iterative_hdfs.py
Log Message:
-----------
HDFS caching abstraction
Commit: d5dbeb4e0fa3166758aced744249e89be71a2891
https://github.com/saga-project/BigJob/commit/d5dbeb4e0fa3166758aced744249e89be71a2891
Author: Andre <andre....@gmail.com>
Date: 2014-03-23 (Sun, 23 Mar 2014)
Changed paths:
A pilot/iterative/clean.sh
M pilot/iterative/iterative_redis.py
M pilot/iterative/iterative_tachyon.py
M setup.py
Log Message:
-----------
further refinements
Commit: 62c1f0702191e6ce29011ab3447a6d39eb159953
https://github.com/saga-project/BigJob/commit/62c1f0702191e6ce29011ab3447a6d39eb159953
Author: Andre <andre....@gmail.com>
Date: 2014-03-23 (Sun, 23 Mar 2014)
Changed paths:
M pilot/iterative/iterative_redis.py
M pilot/iterative/iterative_tachyon.py
Log Message:
-----------
updates for scale-out test
Commit: f6d54f8a89c43abb0443f4439346b1d2e0116798
https://github.com/saga-project/BigJob/commit/f6d54f8a89c43abb0443f4439346b1d2e0116798
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-03-30 (Sun, 30 Mar 2014)
Changed paths:
M examples/hadoop/README.md
Log Message:
-----------
Update README.md
added YARN config
Commit: 70beebce2c6f63abd4b86d3f73fae416628aa3ab
https://github.com/saga-project/BigJob/commit/70beebce2c6f63abd4b86d3f73fae416628aa3ab
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-06-22 (Sun, 22 Jun 2014)
Changed paths:
M examples/hadoop/README.md
A pilot/iterative/clean.sh
M pilot/iterative/iterative_redis.py
M pilot/iterative/iterative_tachyon.py
M setup.py
Log Message:
-----------
Merge branch 'develop' of github.com:saga-project/BigJob into develop
Commit: 7d315a4acb6aab19cc3a752b763927263ae7bd1a
https://github.com/saga-project/BigJob/commit/7d315a4acb6aab19cc3a752b763927263ae7bd1a
Author: Andre <andre....@gmail.com>
Date: 2014-06-22 (Sun, 22 Jun 2014)
Changed paths:
M pilot/filemanagement/hdfs_adaptor.py
M pilot/iterative/iterative_hdfs.py
M pilot/iterative/iterative_redis.py
M pilot/iterative/iterative_tachyon.py
Log Message:
-----------
further tests
Commit: 666bdd2419bdb8776d5f3dc1ba8da110117637f9
https://github.com/saga-project/BigJob/commit/666bdd2419bdb8776d5f3dc1ba8da110117637f9
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-06-22 (Sun, 22 Jun 2014)
Changed paths:
M pilot/iterative/iterative_hdfs.py
Log Message:
-----------
added caching test
Commit: 88969f02ace2d717b89dac31bb10f4b985a09f2a
https://github.com/saga-project/BigJob/commit/88969f02ace2d717b89dac31bb10f4b985a09f2a
Author: Andre Merzky <an...@merzky.net>
Date: 2014-07-13 (Sun, 13 Jul 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
M bootstrap/bigjob-bootstrap.py
M coordination/bigjob_coordination_redis.py
M examples/hadoop/README.md
A examples/pilot-api/example-partial-du-cu-copy.py
M examples/pilot-api/example-pilot-compute-data.py
M pilot/filemanagement/hdfs_adaptor.py
M pilot/filemanagement/ssh_adaptor.py
M pilot/impl/pilotcompute_manager.py
M pilot/impl/pilotdata_manager.py
A pilot/iterative/README.md
A pilot/iterative/__init__.py
A pilot/iterative/clean.sh
A pilot/iterative/iterative_hdfs.py
A pilot/iterative/iterative_redis.py
A pilot/iterative/iterative_tachyon.py
A pilot/iterative/log.txt
A pilot/iterative/rediscluster/__init__.py
A pilot/iterative/rediscluster/rediscluster.py
M setup.py
Log Message:
-----------
Merge branch 'develop' of github.com:saga-project/BigJob into develop
Commit: 852f84e71419890bedf77a7f1f3edc4e678bf2e0
https://github.com/saga-project/BigJob/commit/852f84e71419890bedf77a7f1f3edc4e678bf2e0
Author: Pradeep Kumar Mantha <pma...@login2.stampede.tacc.utexas.edu>
Date: 2014-07-27 (Sun, 27 Jul 2014)
Changed paths:
M bigjob/bigjob_manager.py
M bootstrap/bigjob-bootstrap.py
M examples/pilot-api/example-pilot-api.py
Log Message:
-----------
bootstrap BJ2
Commit: 832db25912701880702db75a0167593e665c1446
https://github.com/saga-project/BigJob/commit/832db25912701880702db75a0167593e665c1446
Author: Andre Luckow <andre....@gmail.com>
Date: 2014-08-31 (Sun, 31 Aug 2014)
Changed paths:
R api/__init__.py
R api/base.py
M bigjob/__init__.py
M bigjob/bigjob_manager.py
M setup.py
Log Message:
-----------
some refactoring
Commit: 291eb34eef478072287dd9675907edb207050110
https://github.com/saga-project/BigJob/commit/291eb34eef478072287dd9675907edb207050110
Author: pmantha <pma...@salesforce.com>
Date: 2014-10-19 (Sun, 19 Oct 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
Log Message:
-----------
get nodes fix
Commit: 2da7592ea610725d214f8e7e46d7a69f016f81c5
https://github.com/saga-project/BigJob/commit/2da7592ea610725d214f8e7e46d7a69f016f81c5
Author: pradeepmantha <prade...@gmail.com>
Date: 2014-10-20 (Mon, 20 Oct 2014)
Changed paths:
M bigjob/bigjob_agent.py
M bigjob/bigjob_manager.py
Log Message:
-----------
Merge pull request #179 from saga-project/get_nodes
get nodes fix
Commit: 9c37d8c4e4790f58346cb8330604e3821235e481
https://github.com/saga-project/BigJob/commit/9c37d8c4e4790f58346cb8330604e3821235e481
Author: pmantha <pma...@salesforce.com>
Date: 2014-10-27 (Mon, 27 Oct 2014)
Changed paths:
M bigjob/__init__.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
modified DU add_files to register files in redis, which are already in DU
Commit: 8fc903782ff346c37804f7df92b5bc3cfde37365
https://github.com/saga-project/BigJob/commit/8fc903782ff346c37804f7df92b5bc3cfde37365
Author: Pradeep <prade...@gmail.com>
Date: 2014-10-29 (Wed, 29 Oct 2014)
Changed paths:
M pilot/coordination/redis_adaptor.py
M setup.py
Log Message:
-----------
too many connections ...
Commit: 767e5a9a637e4f85b41b98f47f8fde4ac4e15319
https://github.com/saga-project/BigJob/commit/767e5a9a637e4f85b41b98f47f8fde4ac4e15319
Author: pmantha <pma...@salesforce.com>
Date: 2014-11-01 (Sat, 01 Nov 2014)
Changed paths:
M bigjob/bigjob_agent.py
M pilot/coordination/redis_adaptor.py
M pilot/impl/pilotdata_manager.py
Log Message:
-----------
input du url restoration for each sj is costly..
Commit: 1e3ecb3a8a4e41b0bb55b31615566530b14197c4
https://github.com/saga-project/BigJob/commit/1e3ecb3a8a4e41b0bb55b31615566530b14197c4
Author: pmantha <pma...@salesforce.com>
Date: 2014-11-01 (Sat, 01 Nov 2014)
Changed paths:
M bigjob/bigjob_agent.py
Log Message:
-----------
minor fix
Compare: https://github.com/saga-project/BigJob/compare/1364772d79bb...1e3ecb3a8a4e
To unsubscribe from these emails, change your notification settings at https://github.com/saga-project/BigJob/settings/notifications
Reply all
Reply to author
Forward
0 new messages