'Module' object has no attribute 'job'

[Jerry]

Hi,

Got an error message "'Module' object has no attribute 'job'"  with Cylc  and failed to run rose stem with ope-2018.07.-0_ops-2018-07.1-release,  tried Cylc 7.5.0, 7.7.0 and 7.8.1,  got the same error message.

Wondering what's the problem and how to fix it ?

Thanks for your time

Regards

 

[Hilary Oliver]

Hi Jerry,

Can you post the Python traceback (which might require running with --debug)?

Hilary

--

---
You received this message because you are subscribed to the Google Groups "cylc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jerry]

Here are the python traceback info.
 

jliu@helios:rose-stem $ rose stem --debug --group=C_XC30 -S CENTRAL_INSTALL=false -S PREBUILDS=false

[INFO] Source tree /home/jliu/roses/ops-2018.07.0_ops-2018.07.1-release added as branch

[INFO] Will run suite from /home/jliu/roses/ops-2018.07.0_ops-2018.07.1-release/rose-stem

[INFO] create: log.20190410T075058Z

[INFO] delete: log

[INFO] symlink: log.20190410T075058Z <= log

[INFO] log.20190410T065125Z.tar.gz <= log.20190410T065125Z

[INFO] delete: log.20190410T065125Z/

[INFO] create: log/suite

[INFO] create: log/rose-conf

[INFO] symlink: rose-conf/20190410T075058-run.conf <= log/rose-suite-run.conf

[INFO] symlink: rose-conf/20190410T075058-run.version <= log/rose-suite-run.version

[INFO] export CYLC_VERSION=7.5.0

[INFO] export ROSE_ORIG_HOST=helios

[INFO] export ROSE_VERSION=2018.05.0

[INFO] REGISTER ops-2018.07.0_ops-2018.07.1-release: /home/jliu/cylc-run/ops-2018.07.0_ops-2018.07.1-release

Traceback (most recent call last):

  File "/scratch/singadm/pkg_helios/Python/2.7.10/lib/python2.7/runpy.py", line 162, in _run_module_as_main

    "__main__", fname, loader, pkg_name)

  File "/scratch/singadm/pkg_helios/Python/2.7.10/lib/python2.7/runpy.py", line 72, in _run_code

    exec code in run_globals

  File "/scratch/singadm/rose/2018.05.0/lib/python/rose/stem.py", line 494, in <module>

    main()

  File "/scratch/singadm/rose/2018.05.0/lib/python/rose/stem.py", line 482, in main

    sys.exit(runner(opts, args))

  File "/scratch/singadm/rose/2018.05.0/lib/python/rose/run.py", line 199, in run

    return self.run_impl(opts, args, uuid, work_files)

  File "/scratch/singadm/rose/2018.05.0/lib/python/rose/suite_run.py", line 327, in run_impl

    opts.debug_mode)

  File "/scratch/singadm/rose/2018.05.0/lib/python/rose/suite_engine_procs/cylc.py", line 126, in cmp_suite_conf

    self.popen.run_simple(*command, stdout_level=Event.V)

  File "/scratch/singadm/rose/2018.05.0/lib/python/rose/popen.py", line 221, in run_simple

    args, ret_code, stdout, stderr, kwargs.get("stdin"))

rose.popen.RosePopenError: cylc validate -v -o /tmp/tmpQe_JAA --debug --strict ops-2018.07.0_ops-2018.07.1-release # return-code=1, stderr=

Traceback (most recent call last):

  File "/scratch/singadm/cylc/cylc-7.5.0/bin/cylc-validate", line 131, in <module>

    main()

  File "/scratch/singadm/cylc/cylc-7.5.0/bin/cylc-validate", line 83, in main

    mem_log_func=profiler.log_memory)

  File "/scratch/singadm/cylc/cylc-7.5.0/lib/cylc/config.py", line 109, in get_inst

    vis_start_string, vis_stop_string, mem_log_func)

  File "/scratch/singadm/cylc/cylc-7.5.0/lib/cylc/config.py", line 182, in __init__

    output_fname=output_fname)

  File "/scratch/singadm/cylc/cylc-7.5.0/lib/cylc/cfgspec/suite.py", line 566, in get_inst

    cls._INSTANCES[fpath].loadcfg(fpath, "suite definition")

  File "/scratch/singadm/cylc/cylc-7.5.0/lib/parsec/config.py", line 68, in loadcfg

    sparse = parse(rcfile, self.output_fname, self.tvars)

  File "/scratch/singadm/cylc/cylc-7.5.0/lib/parsec/fileparse.py", line 313, in parse

    flines = read_and_proc(fpath, template_vars)

  File "/scratch/singadm/cylc/cylc-7.5.0/lib/parsec/fileparse.py", line 268, in read_and_proc

    flines = jinja2process(flines, fdir, template_vars)

  File "/scratch/singadm/cylc/cylc-7.5.0/lib/parsec/jinja2support.py", line 67, in jinja2process

    env.filters[fname] = getattr(module, fname)

AttributeError: 'module' object has no attribute 'job'

  

[Hilary Oliver]

OK, I would say that is caused by (in the suite definition) use of a custom Jinja2 filter in which the filter function name is not the same as the module name.

If you use a custom filter such as {{ foo | myfilter }} ... then Cylc expects to find a function called "myfilter" inside a module "myfilter.py".

(However, we should generate a more user-friendly error message for this scenario).

Hilary

--

[Jerry]

Hi Hilary,

Does it mean the suite is 'incomplete' ? 

There are lot of custom Jinja2 filters in the suite, such as {{task_name | job }}.   And there is a python module file called job.py in directory rose-stem/Jinja2Filters, just python could not pick up the module file path. 

So how to get Cylc pick up the module file path Jinja2Filters for python ?

Thanks for your time

Regards

Jerry

On Wednesday, April 10, 2019 at 5:20:36 PM UTC+8, Hilary Oliver wrote:

OK, I would say that is caused by (in the suite definition) use of a custom Jinja2 filter in which the filter function name is not the same as the module name.

If you use a custom filter such as {{ foo | myfilter }} ... then Cylc expects to find a function called "myfilter" inside a module "myfilter.py".

(However, we should generate a more user-friendly error message for this scenario).

Hilary

On Wed, 10 Apr 2019 at 19:57, Jerry <jerr...@gmail.com> wrote:

Here are the python traceback info.
 

jl...@helios:rose-stem $ rose stem --debug --group=C_XC30 -S CENTRAL_INSTALL=false -S PREBUILDS=false

To unsubscribe from this group and stop receiving emails from it, send an email to cy...@googlegroups.com.

[Hilary Oliver]

Hi  Jerry,

I believe your error message indicates that the "job" module was found, but `job.py` does not define a function called "job".

Cylc automatically puts <suite-dir>/Jinja2filters/ in the Python path.  If your "job" filter module does define a "job" function, then perhaps you have another "job" module earlier in your Python path?

(although that seems unlikely ... I haven't checked, but Cylc presumably puts custom filter directories at the front of the search path, not at the back).

Hilary

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.

[Jerry]

Hi Hilary,

Thanks for your tips.

There is another "job" module earlier in the Python Path which was installed in the system default site-package directory.

I think Cylc may put custom filter directories at the end of the search path.

lib/parsec/jinja2support.py, 

for fdir in [ os.path.join(os.environ['CYLC_DIR'], 'lib', 'Jinja2Filters'),

           os.path.join(dir_,'Jinja2Filters'),

           os.path.join(os.environ['HOME'],'.cylc', 'Jinja2Filters')]:

     if os.path.isdir(fdir):

           sys.path.append(os.path.abspath(fdir))

Or I misunderstood ?

[Hilary Oliver]

Hi Jerry,

OK, that appears to be the problem!

I can't think why we are appending to the search path, presumably it should be pre-pending; I'll post a bug report.

In the meantime, I guess the workaround is to modify the code you found to prepend the filters directory to the path, OR change the name of the "job" module (and the function within it, and any references to it in the suite definition).

Regards,

Hilary

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.

[Hilary Oliver]

Hi again Jerry,

https://github.com/cylc/cylc/issues/3109

Maybe you'd like to submit a code fix for this?? (by Pull Request on GitHub)

Hilary

[Jerry]

Hi Hilary,

Here are my fix for parsec/jinja2support.py

# pre-pending instead appending

line 57 :  orig_sys_path = sys.path

line 64:   sys.path = [os.path.abspath(fdir)] + orig_sys_path

How to submit the fix to github ?

Regards

[Hilary Oliver]

Hi Jerry,

Thanks, I see you've updated the Issue on GitHub. We can take it from here - custom filters will be pre-prended to the Python search path in next release (7.8.2).

Hilary

To unsubscribe from this group and stop receiving emails from it, send an email to cylc+uns...@googlegroups.com.