Cannot Create HIT (__init()__ error)

[Jorja Shires]

Hi,

I was trying to start a HIT and encountered an error message:

Response(Operation: create_hit | Status: error | Exception: __init__() takes 1 positional argument but 2 were given)

I tried creating a hit in live mode and sandbox mode. I went to older experiments (in entirely different folders) that worked fine last week and got the same error message. I restarted psiturk and terminal. No matter what I did, I got the same error message. 

Here was what I was entering to the hit create:

[psiTurk server:on mode:sdbx #HITs:0]$ hit create

number of participants? 1

reward per HIT? 1.00

duration of hit (in hours, it can be decimals)? 1

*****************************

    Max workers: 1

    Reward: $1.00

    Duration: 1.0 hours

    Fee: $0.20

    ________________________

    Total: $1.20

Create sandbox HIT [y/n]? y

I also tried creating a hit in this method:

hit create 1 1.00 1

I got the same error message. Psiturk version 2.3.11. Mac Catalina ver 10.15.17

[Dave Eargle]

Whaaa? That version has been in the wild since October last year, why in the world would something be wrong all of a sudden I wonder?

--
You received this message because you are subscribed to the Google Groups "PsiTurk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psiturk+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/2171d630-9d19-4bd8-8ce6-b03e6db0c33an%40googlegroups.com.

[Jorja Shires]

I'm glad I'm not the only who finds it weird. It was working fine last Wednesday. I could try reinstalling psiturk. 

[Dave Eargle]

Yeah can you do that, and use a fresh virtual environment? Heads up that psiturk v3 has been released, so you'll want to pin to 2.3.11 

To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/15ed9d60-d55f-4580-a5ec-985a0d55e935n%40googlegroups.com.

[Jorja Shires]

Okay, thanks for the heads up. I installed it in a fresh environment and got the same error. 

Something really strange happened when I first tried to uninstall it. According to my computer, despite the fact I had been using psiturk to debug since about 9 AM this morning, psiturk was not installed. I installed it, uninstalled, and then made the virtual environment and installed in that environment.  

[Dave Eargle]

Todd says he is going to install 2.3.11 tomorrow and track down what's going wrong on the psiturk-org server. The python code won't give any more insight.

To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/e627d4ea-84b8-45d6-8c5d-f2d4ebdae64an%40googlegroups.com.

[Jorja Shires]

Great, let me know if there is anything I can do on my end. Thanks to both you and Todd. Psiturk has been a life saver, especially during COVID. 

[Dave Eargle]

@todd confirmed, the error being thrown is right here (first screenshot). I added the `message=`. I attached a debugger and the call in the second screenshot is returning a 500. That's to https://api.psiturk.org/api/sandboxad . The content that it's posting looks normal, and my credentials pass when I manually put them into http://api.psiturk.org/api/ad . 

To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/058bff53-7fac-4b8f-83cb-58dd2a08dbb9n%40googlegroups.com.

[Jorja Shires]

I checked my psiturk key and access id. They were right, but I went ahead and had them regenerate new ones and updated the file. I tried to create a hit and got the same error as before. 

[Dave Eargle]

I've put way too much time into this, but api.psiturk.org lives on a dreamhost server, and I think they recently updated their security filters in a way that flags the ad html. On a guess I created a version that doesn't have any of the html comments, and I think it posted successfully. It's here: https://gist.github.com/deargle/b5fedb239415b66b58a75f0dcaf038bf

I need to pivot to other things for a while.

To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/72ee06cd-15b3-484a-afb9-80abfe308278n%40googlegroups.com.

[Dave Eargle]

Jk I think it's actually the json parser for some reason no longer working. I had made another change in addition to removing the html which led to things magically working. I had changed requests.post to use json=content instead of data=json.dumps(content), and that in combination with the above works. I made the change here: https://github.com/NYUCCL/psiTurk/blob/20383d9a2f123acf4d27c30069da3b088c16992a/psiturk/psiturk_org_services.py#L90-L95 

I have no idea why things would suddenly not json.dumps to valid html. I cry.

[Dave Eargle]

Third update within 60 seconds -- using json= instead of content=json.dumps sets the content-type header to application/json . I tested with manually adding the content-type: application/json header and back to using the data=json.dumps(), which I think is the only difference, and it worked, so I'm back to thinking it's an OWASP catch that handles application/json headers differently, dunno. Either way, I cry.

[Jorja Shires]

Thank you so much! Switching to json = content worked! 

[Dave Eargle]

It did?! Did you also use my gist ad? Because I tried with json = with the original ad and it's not working, suggesting that those two things need to happen -- remove all html comments

from the ad for some reason, and then set the application/json header. 

To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/8a1c113a-a7b3-4630-9870-2d8c0d5a80d0n%40googlegroups.com.

[Jorja Shires]

Yes, I used your gist ad and made the json = content switch. 

I just tested it on a different experiment (without the gist ad, but with the json = content) and it threw the same error. Both changes are necessary. 

[David Halpern]

Thanks so much for looking into this Dave and Jorja. I think I'm a bit unclear on how to make the "json = content" change? Did you edit psiturk_org_services.py and then reinstall psiturk? or is it something in the ad html that I'm missing?

[Jorja Shires]

1. I opened psiturk_org_services.py, made the change from data=json.dumps(content) to json=content. The exact line that is changed is highlighted when you click on the link Dave sent. I saved and close the py file. No reinstall needed.

2. In ad.html, all comments were erased. In the gist ad, you can see that it has no comments. I went back to my old ad.html and erased all the comments myself (it wasn't much) and saved ad.html. 

After those 2 changes, it successfully created a HIT. 

[Dave Eargle]

This assumes that you're installed psiturk via git clone and cd psiturk and pip install -e .

I'm scrambling to get ready to teach here in a few minutes, i put it off because was obsessed with this bug, later I'll try to help more

To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/0a3c0bab-a281-4ddf-af45-bc2ecdfd9c66n%40googlegroups.com.

[David Halpern]

Thanks! I didn't realize that json was a parameter of requests.post and thought it was referring to the json package which is why i was confused... I think I understand what to try now but any further insight would be very welcome Dave!

You received this message because you are subscribed to a topic in the Google Groups "PsiTurk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/psiturk/cYjEmLXy0jk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to psiturk+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/CAKL%3D07ROWuzuxeZ2JoXExozNam%2B32%2BMddDNTbxJ3eaPR7HOpWQ%40mail.gmail.com.

[David Halpern]

Can confirm that these changes worked for me too!

[Evelyn Morris]

Hello,

I encountered the same error on the 16th. I didn't have psiturk installed via that method, so I created a new virtual environment and installed psiturk via the git-clone method. However, now I am getting a new error when I attempt to turn the server on:

Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "/home/usernameD.Brown.Edu/anaconda3/lib/python3.7/site-packages/gunicorn/arbiter.py", line 242, in handle_chld
    self.reap_workers()
  File "/home/user...@AD.Brown.Edu/anaconda3/lib/python3.7/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
2021-03-19T13:35:28Z <Greenlet at 0x7fb66c1fdb38: <bound method Arbiter.handle_chld of <gunicorn.arbiter.Arbiter object at 0x7fb66e1bfe80>>(17, None)> failed with HaltServer

When I attempt to create a hit, I get this error:

Error during hit creation.
Response(Operation: create_hit | Status: error | Exception: PsiturkException: missing ad_url config var `ad_url_domain`)

I've attached screenshots as well. Do you know what might be going on? I'm new to psiturk, so apologies if I am missing something obvious.

Psiturk version 3.0.4. Windows.

Thanks,

Evelyn

[Dave Eargle]

If you still want to use the psiturk ad server, downgrade to the highest psiturk v2 on pip. I think that would be `pip install psiturk~=2.3`

Otherwise, check out the migration guide from 2 to 3. https://psiturk.readthedocs.io/en/latest/migrating.html

To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/7ed5f421-dd1c-4626-8080-fddbc72229b9n%40googlegroups.com.

[Brian Drwecki]

Hi Dave, Thanks for all your hard work helping us folks out. 

I see you created version 2.3.12 to fix this issue. But, I couldn't install it with pip install psiturk ~=2.3.    2.3.11 is the most recent version on https://pypi.org/project/PsiTurk/#history. 

If any other readers are struggling: Dave's fix is posted here: https://github.com/NYUCCL/psiTurk/releases/tag/v2.3.12; I downloaded the zip file, navigated to its context via the terminal. I navigated inside the psiTurk-2.3.12 folder 

 then  I typed "sudo pip install -e ." [the period is necessary; the quotes are not] It installed and it let me post a hit. 

FYI: my system has python 3.9.0 [it failed with 2.7.X]; Mac Big Sur; and now 2.3.12 psiturk. 

Thanks Dave, 

[Dave Eargle]

That's odd, usually a release on github is picked up by Travis CI and pushed to pip automatically. I'll look into it later. Thanks for the instructions to others.

The other somewhat easier option is to a `git clone` of psiturk, `cd psiturk`, `git checkout python2` (that's the branch that hotfixes from psiturk 2 are released from), then `pip install -e .` , or just `pip install .`

The -e flag will let you hot-edit the files in the git clone, and changes will be picked up on the next launch of psiturk. Otherwise, without it, the package installs to a site-packages folder.

To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/843102f5-ca05-431c-baae-071668f916ccn%40googlegroups.com.

[Dave Eargle]

I got v2.3.12 to push to pip. So `pip install psiturk==2.3.12` should work now.

[Evelyn Morris]

Hello,

Thank you so much for all the help! I'm still having some trouble with upgrading from psiturk 2.3.3 (in which I encounter the original problem on this thread) to psiturk 2.3.12. I initially tried to install it and got an error message saying that it failed to build ciso8601 and cryptography. The error message recommended upgrading pip, so I upgraded pip from 18.1 to 21.01. Now I am getting the following error message.

Any recommendations for how I can proceed? Thank you again for all the help - we are running up against a deadline for data collection and I've been having a hard time figuring this out.

  ERROR: Command errored out with exit status 1:
   command: /home/ username @AD.Brown.Edu/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-y9myxaoy/ciso8601_f6d40623bbae44ebbcce21e8f1196bfc/setup.py'"'"'; __file__='"'"'/tmp/pip-install-y9myxaoy/ciso8601_f6d40623bbae44ebbcce21e8f1196bfc/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-v_nkt4cr
       cwd: /tmp/pip-install-y9myxaoy/ciso8601_f6d40623bbae44ebbcce21e8f1196bfc/
  Complete output (15 lines):
  running bdist_wheel
  running build
  running build_py
  package init file 'ciso8601/__init__.py' not found (or not a regular file)
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/ciso8601
  copying ciso8601/__init__.pyi -> build/lib.linux-x86_64-3.7/ciso8601
  copying ciso8601/py.typed -> build/lib.linux-x86_64-3.7/ciso8601
  running build_ext
  building 'ciso8601' extension
  creating build/temp.linux-x86_64-3.7
  gcc -pthread -B /home/ username @AD.Brown.Edu/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DCISO8601_VERSION=2.1.3 -I/home/ username @AD.Brown.Edu/anaconda3/include/python3.7m -c module.c -o build/temp.linux-x86_64-3.7/module.o
  unable to execute 'gcc': No such file or directory
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for ciso8601
  Running setup.py clean for ciso8601
  Building wheel for psutil (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/ username @AD.Brown.Edu/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-y9myxaoy/psutil_59ef863187a84c6ea9258823b2a59fcc/setup.py'"'"'; __file__='"'"'/tmp/pip-install-y9myxaoy/psutil_59ef863187a84c6ea9258823b2a59fcc/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-v4p3u6iz
       cwd: /tmp/pip-install-y9myxaoy/psutil_59ef863187a84c6ea9258823b2a59fcc/
  Complete output (41 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/psutil
  copying psutil/_common.py -> build/lib.linux-x86_64-3.7/psutil
  copying psutil/_compat.py -> build/lib.linux-x86_64-3.7/psutil
  copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.7/psutil
  copying psutil/__init__.py -> build/lib.linux-x86_64-3.7/psutil
  copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.7/psutil
  copying psutil/_psposix.py -> build/lib.linux-x86_64-3.7/psutil
  copying psutil/_psosx.py -> build/lib.linux-x86_64-3.7/psutil
  copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.7/psutil
  copying psutil/_psaix.py -> build/lib.linux-x86_64-3.7/psutil
  copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.7/psutil
  creating build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_connections.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/runner.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_unicode.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_system.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/__init__.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_process.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/__main__.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-3.7/psutil/tests
  copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-3.7/psutil/tests
  running build_ext
  building 'psutil._psutil_linux' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/psutil
  gcc -pthread -B /home/ username @AD.Brown.Edu/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 -DPSUTIL_LINUX=1 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -I/home/ username @AD.Brown.Edu/anaconda3/include/python3.7m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.7/psutil/_psutil_common.o
  unable to execute 'gcc': No such file or directory
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for psutil
  Running setup.py clean for psutil
  Building wheel for setproctitle (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/ username @AD.Brown.Edu/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-y9myxaoy/setproctitle_543ff08594264f0d8f238576764bc333/setup.py'"'"'; __file__='"'"'/tmp/pip-install-y9myxaoy/setproctitle_543ff08594264f0d8f238576764bc333/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-oga_3ibj
       cwd: /tmp/pip-install-y9myxaoy/setproctitle_543ff08594264f0d8f238576764bc333/
  Complete output (10 lines):
  running bdist_wheel
  running build
  running build_ext
  building 'setproctitle' extension
  creating build
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/src
  gcc -pthread -B /home/ username @AD.Brown.Edu/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DSPT_VERSION=1.1.10 -DHAVE_SYS_PRCTL_H=1 -I/home/ username @AD.Brown.Edu/anaconda3/include/python3.7m -c src/setproctitle.c -o build/temp.linux-x86_64-3.7/src/setproctitle.o
  unable to execute 'gcc': No such file or directory
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for setproctitle
  Running setup.py clean for setproctitle
Failed to build ciso8601 psutil setproctitle
Installing collected packages: six, zipp, urllib3, typing-extensions, smmap, python-dateutil, pycparser, jmespath, docutils, smmap2, pyparsing, MarkupSafe, importlib-metadata, cffi, botocore, Werkzeug, wcwidth, ua-parser, text-unidecode, s3transfer, pyperclip, py, pluggy, packaging, more-itertools, Jinja2, itsdangerous, idna, gitdb2, cryptography, click, chardet, certifi, attrs, atomicwrites, user-agents, SQLAlchemy, setproctitle, requests, pytest, pyOpenSSL, psutil, mock, gunicorn, gnureadline, GitPython, fuzzywuzzy, future, Flask, Faker, docopt, cmd2, ciso8601, boto3, psiturk
    Running setup.py install for setproctitle ... error
    ERROR: Command errored out with exit status 1:
     command: /home/ username @AD.Brown.Edu/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-y9myxaoy/setproctitle_543ff08594264f0d8f238576764bc333/setup.py'"'"'; __file__='"'"'/tmp/pip-install-y9myxaoy/setproctitle_543ff08594264f0d8f238576764bc333/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-u3jyexko/install-record.txt --single-version-externally-managed --compile --install-headers /home/ username @AD.Brown.Edu/anaconda3/include/python3.7m/setproctitle
         cwd: /tmp/pip-install-y9myxaoy/setproctitle_543ff08594264f0d8f238576764bc333/
    Complete output (10 lines):
    running install
    running build
    running build_ext
    building 'setproctitle' extension
    creating build
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/src
    gcc -pthread -B /home/ username @AD.Brown.Edu/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DSPT_VERSION=1.1.10 -DHAVE_SYS_PRCTL_H=1 -I/home/ username @AD.Brown.Edu/anaconda3/include/python3.7m -c src/setproctitle.c -o build/temp.linux-x86_64-3.7/src/setproctitle.o
    unable to execute 'gcc': No such file or directory
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/user...@AD.Brown.Edu/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-y9myxaoy/setproctitle_543ff08594264f0d8f238576764bc333/setup.py'"'"'; __file__='"'"'/tmp/pip-install-y9myxaoy/setproctitle_543ff08594264f0d8f238576764bc333/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-u3jyexko/install-record.txt --single-version-externally-managed --compile --install-headers /home/ username @AD.Brown.Edu/anaconda3/include/python3.7m/setproctitle Check the logs for full command output.

[Dave Eargle]

That error isn't related to anything psiturk-specific -- rather, anaconda and pip are not great friends. I recommend creating a fresh conda environment, installing pip into that environment via conda repositories, then running "pip install psiturk==2.3.12"

See https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#using-pip-in-an-environment but do "conda create..." for the first command instead of "conda install..."

To view this discussion on the web visit https://groups.google.com/d/msgid/psiturk/f1cc4c1d-bfce-4e48-8050-cf9a489cf83bn%40googlegroups.com.