Internal Server Error

[Aman Verma]

Hi: When I am running beat dataformats list --remote I am getting following error: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

So, I thought of changing the config file by setting my account's token. But when I am trying to access the same on BEAT web, I am getting Internal Server Error. I am attaching a screenshot of the error.

Regards
Aman Verma

[SG]

Hi,

Can you try again ?

I just tested system access and it seems to work properly both on the command line and the web site.

Also, double check the URL you are using is the correct one: https://www.idiap.ch/software/beat/platform

Best regards

SG

[Aman Verma]

Hi: The same error persists for both command line and the web site. Since, I am accessing the network from India, is my IP creating this mess? I am attaching screenshots of the error.
rs. 

[Aman Verma]

Hi: Just being more specific, I am trying to run LivDet-Iris 2020 experiment. Is this some domain-inherent issue or a generalized one?

Regards
Aman Verma

[SG]

Hi,

Please use https://www.idiap.ch/software/beat/platform.

To ensure you are doing that you can do `beat -m https://www.idiap.ch/software/beat/platform dataformat list --remote`

You should also call `beat config set platform https://www.idiap.ch/software/beat/platform` so it is saved in a configuration file so you don't have to use -m on each invocation. It will also be useful when using beat editor.

Best regards

Samuel

[Aman Verma]

Hi: I tried this and this works fine. Thanks a ton!!

If there is any further complication: I'll post it.

Regards
Aman Verma

[Aman Verma]

Hi: I tried running beat algorithms pull  amohammadi/livdet_analyzer/1. I am getting following error: 

I am sharing the entire traceback: 

Traceback (most recent call last):
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 909, in resolve_from_url
    document = self.store[url]
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/_utils.py", line 28, in __getitem__
    return self.store[self.normalize(uri)]
KeyError: 'file:///home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/core/schema/dataformat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/urllib/request.py", line 1521, in open_local_file
    return addinfourl(open(localfile, 'rb'), headers, origurl)
IsADirectoryError: [Errno 21] Is a directory: '/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/core/schema/dataformat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 912, in resolve_from_url
    document = self.resolve_remote(url)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 1018, in resolve_remote
    with urlopen(uri) as url:
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/urllib/request.py", line 1484, in file_open
    return self.open_local_file(req)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/urllib/request.py", line 1523, in open_local_file
    raise URLError(exp)
urllib.error.URLError: <urlopen error [Errno 21] Is a directory: '/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/core/schema/dataformat'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/bin/beat", line 11, in <module>
    sys.exit(main())
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/cmdline/decorators.py", line 94, in _decorator
    value = view_func(*args, **kwargs)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/cmdline/algorithms.py", line 391, in pull
    return pull_impl(webapi, ctx.meta["config"].path, name, force, 0, {}, {})
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/cmdline/algorithms.py", line 121, in pull_impl
    obj = algorithm.Algorithm(prefix, name)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/core/algorithm.py", line 187, in __init__
    super(Algorithm, self).__init__(prefix, data, dataformat_cache, library_cache)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/backend/python/algorithm.py", line 475, in __init__
    self._load(name, dataformat_cache, library_cache)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/core/algorithm.py", line 275, in _load
    self._validate_required_dataformats(dataformat_cache)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/core/algorithm.py", line 348, in _validate_required_dataformats
    result_type, dataformat_cache
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/backend/python/algorithm.py", line 529, in _update_dataformat_cache
    thisformat = self.dataformat_klass(self.prefix, type_name)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/core/dataformat.py", line 116, in __init__
    super(DataFormat, self).__init__(prefix, data, parent, dataformat_cache)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/backend/python/dataformat.py", line 166, in __init__
    self._load(data, dataformat_cache)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/core/dataformat.py", line 149, in _load
    self.data, self.errors = schema.validate("dataformat", data)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/core/schema/__init__.py", line 186, in validate
    for k in sorted(validator.iter_errors(data), key=lambda e: e.path)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 288, in iter_errors
    for error in errors:
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/_validators.py", line 26, in patternProperties
    v, subschema, path=k, schema_path=pattern,
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 305, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 288, in iter_errors
    for error in errors:
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/_validators.py", line 383, in oneOf
    errs = list(validator.descend(instance, subschema, schema_path=index))
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 305, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 288, in iter_errors
    for error in errors:
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/_validators.py", line 298, in ref
    yield from validator.descend(instance, resolved)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 305, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 288, in iter_errors
    for error in errors:
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/_legacy_validators.py", line 111, in items_draft3_draft4
    item, subschema, path=index, schema_path=index,
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 305, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 288, in iter_errors
    for error in errors:
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/_validators.py", line 383, in oneOf
    errs = list(validator.descend(instance, subschema, schema_path=index))
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 305, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 288, in iter_errors
    for error in errors:
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/_validators.py", line 294, in ref
    scope, resolved = validator.resolver.resolve(ref)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 898, in resolve
    return url, self._remote_cache(url)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/jsonschema/validators.py", line 914, in resolve_from_url
    raise exceptions.RefResolutionError(exc)
jsonschema.exceptions.RefResolutionError: <urlopen error [Errno 21] Is a directory: '/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/core/s

[SG]

Hi,

Seems there where updates to jsonschema since the last release of beat.core that modified some behaviour. Please downgrade it to version 3 using something like `conda install jsonschema=3`.

Best regards

SG

[SG]

Additional note: beat.core requires jsonschema < 4.0 so there's some glitch somewhere else that allowed this later version of be installed.

Best regards

SG

[Aman Verma]

Hi: There is another error that shoots up on hitting 'beat experiments pull amohammadi/amohammadi/livdet/1/livdet'. But the same works well for: 'beat experiments fork anjos/anjos/livdet-lda/1/livdet-2013-biometrika-test'

I am sharing its traceback:

retrieving `experiments/amohammadi/amohammadi/livdet/1/livdet'...
    retrieving `toolchains/amohammadi/livdet/1'...
    retrieving `databases/livdet-iris-2020/1'...

Traceback (most recent call last):
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/bin/beat", line 11, in <module>
    sys.exit(main())
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 1130, in __ca                                                                                                                        ll__
    return self.main(*args, **kwargs)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 1657, in invo                                                                                                                        ke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 1657, in invo                                                                                                                        ke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 1404, in invo                                                                                                                        ke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/core.py", line 760, in invok                                                                                                                        e
    return __callback(*args, **kwargs)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/click/decorators.py", line 26, in                                                                                                                         new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/cmdline/decorators.py", line                                                                                                                         94, in _decorator
    value = view_func(*args, **kwargs)

  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/cmdline/experiments.py", line                                                                                                                         820, in pull
    return pull_impl(webapi, config.path, names, force, 0, {})
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/cmdline/experiments.py", line                                                                                                                         500, in pull_impl
    webapi, prefix, databases, force, indentation, format_cache
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/cmdline/databases.py", line 3                                                                                                                        76, in pull_impl
    indentation,
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/cmdline/common.py", line 853,                                                                                                                         in pull
    data = fetch_object(webapi, asset_type, name, fields)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/cmdline/common.py", line 778,                                                                                                                         in fetch_object
    return webapi.get(url)
  File "/home/rupesh/anaconda3/envs/beat_irpad_1/lib/python3.7/site-packages/beat/cmdline/webapi.py", line 101,                                                                                                                         in get
    raise WebAPIError("GET", url, answer)
beat.cmdline.webapi.WebAPIError: GET error 404: https://www.idiap.ch/software/beat/platform/api/v1/databases/li                                                                                                                        vdet-iris-2020/1/?object_format=string&fields=declaration,code,description
Error:<HTML>
<HEAD>
<TITLE>Page Not Found (404)</TITLE>
<STYLE TYPE="text/css">
BODY,TABLE {FONT:12px sans-serif;BACKGROUND:#FFFFFF;}
A {TEXT-DECORATION:none;COLOR:#0086FF;}
A:hover {TEXT-DECORATION:underline;}
IMG {BORDER:none;}
</STYLE>
</HEAD>
<BODY STYLE="WIDTH:100%;">
<TABLE STYLE="WIDTH:100%;MARGIN-TOP:25px;"><TR>
<TD STYLE="TEXT-ALIGN:center;VERTICAL-ALIGN:middle">
<DIV STYLE="POSITION:relative;MARGIN-TOP:50px;MARGIN-BOTTOM:20px;">
<IMG ALT="Idiap Research Institute" SRC="/GLOBAL/idiap.png" WIDTH="320" HEIGHT="85">
<DIV STYLE="POSITION:relative;LEFT:155px;TOP:-50px;"><IMG SRC="/GLOBAL/404.png" WIDTH="64" HEIGHT="64"></DIV>
<DIV STYLE="POSITION:relative;TOP:-50px;"><P><A HREF="http://www.idiap.ch">Idiap Research Institute</A></P></DI                                                                                                                        V>
</DIV>
<DIV STYLE="WIDTH:800px;HEIGHT:400px;MARGIN:auto;">
<P><B>Page Not Found</B></P>
<P>The page you requested could not be found. Please check/update your URL.<BR/>
Please call Idiap helpdesk if you need any help or further information: +41 27 721 77 77</P>
</DIV>
</TD>
</TR></TABLE>
</BODY>
</HTML>

Regards
Aman Verma

[Aman Verma]

Clarification: I was referring to ' beat experiments pull anjos/anjos/livdet-lda/1/livdet-2013-biometrika-test' and not ' beat experiments fork anjos/anjos/livdet-lda/1/livdet-2013-biometrika-test'.

[SG]

That's because amohammadi/amohammadi/livdet/1/livdet uses a non-public dataset hence the error you are getting.

What you can do is edit the experiment to use version 2 of the database which is public.

Best regards

SG

[Aman Verma]

Hi: Thanks for the quick response. A couple of quick queries:
1) Can you please share the link to the version 2 (from BEAT web)
2) Where can I edit the experiment, since I am not able to pull it.

Regards
Aman Verma

[SG]

Hi,

1) https://www.idiap.ch/software/beat/platform/databases/livdet-iris-2020/2/

2) My bad, I forgot that the order was downloading dependencies first. However, this experiment is based on a really simple toolchain that is composed of three blocks:

    1) Input database

    2) Algorithmic block

    3) Analyser

    So what you can do is to create a new experiment using the toolchain of that one and configure it using the information from the detail page and using version 2 of the livdet-iris-2020 database

Best regards

SG

[Aman Verma]

Hi: I am asking a very silly question, but can you let me know where can I club these three blocks together (the data, the algorithm and the analyzer) as an experiment?

Regards
Aman Verma

[Aman Verma]

Hi: I have got the things sorted, just can you let me know, how can I pass my algorithm (tensorflow model), i.e. python script, in the toolchain?

Regards
Aman Verma

[SG]

Hi,

You don't pass it to the toolchain. A toolchain is a blueprint that defines an experiment structure.

First, you need to create an Algorithm that will contain your code.

Next, create a new experiment based on that toolchain and while configuring the various blocks, select your newly create Algorithm asset for the processing block.

Best regards

SG

[Aman Verma]

Hi: Thanks for the response. Needed a clarification: the algorithm is encapsulated inside the .json file, do I need to specify path to python script inside that file? Can you share some example?

Regards
Aman Verma

[SG]

Hi,

An algorithm is a combination of its definition (the JSON file), the code (the python file) and optionally documentation (the rst file). All these files resides adjacent to each other.

Note that an Algorithm has to follow a certain structure in order to be used on the platform.

Based on your previous posted images, you have already cloned the tutorial prefix which contains examples of the various assets that are used to build a BEAT experiment. You can also pull algorithms from the platform to inspect them.

Best regards

SG