Error when trying to insert csv values into sql table.(no orm allowed)

1,171 views
Skip to first unread message

Cravan

unread,
Jun 13, 2019, 6:15:24 AM6/13/19
to sqlalchemy

I'm getting a weird error code when I try to store values from a csv into an sql table in a movie review assignment.

I have already edited my apostrophes and spacing and looked up examples from google to try and resolve my error to no avail. I also ensured that i defined DATABASE_URL properly. Sorry for the long traceback error at the end :P Please note that my csv values are stored in lists in each cell. They are arranged in a single column such as

The Lego Movie;2014;100;tt1490017;7.8

This is my main code

import csv
import sys
import os
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy import create_engine
from flask import Flask, render_template, request, session
from flask_sqlalchemy import SQLAlchemy
from flask_session import Session

engine = create_engine(os.getenv("DATABASE_URL")) # database engine object from SQLAlchemy that manages connections to the database,# DATABASE_URL is an environment variable that indicates where the database lives

def main():
    f = open("movies.csv","r")
    reader = csv.reader(f, delimiter=';')
    for i, row in enumerate(reader): # loop gives each column a name
        if i == 0:
            continue
        title = row[0]
        year = int(row[1])
        runtime = int(row[2])
        imdbID = row[3]
        imdbRating = float(row[4])
        engine.execute('INSERT INTO movies("title","year","runtime","imdbID","imdbRating") VALUES ((title), (year), (runtime), (imdbID), (imdbRating))',
        {"title": title, "year": year, "runtime": runtime, "imdbID": imdbID, "imdbRating": imdbRating })
    engine.commit() # transactions are assumed, so close the transaction finished
if __name__ == "__main__":
    main()

SQL code:

CREATE TABLE movies (
      "title" SERIAL PRIMARY KEY,
      "year" INTEGER NOT NULL,
      "runtime" INTEGER NOT NULL,
      "imdbID" VARCHAR NOT NULL,
      "imdbRating" INTEGER NOT NULL
  );

New error code:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/engine/default.py", line 550, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedColumn: column "title" does not exist
LINE 1: ...,"year","runtime","imdbID","imdbRating") VALUES ((title), (y...
                                                             ^
HINT:  There is a column named "title" in table "movies", but it cannot be re
ferenced from this part of the query.


The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "import.py", line 26, in <module>
    main()
  File "import.py", line 23, in main
    {"title": title, "year": year, "runtime": runtime, "imdbID": imdbID, "imd
bRating": imdbRating })
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/engine/base.py", line 2166, in execute
    return connection.execute(statement, *multiparams, **params)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/engine/base.py", line 982, in execute
    return self._execute_text(object_, multiparams, params)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/engine/base.py", line 1155, in _execute_text
    parameters,
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    e, statement, parameters, cursor, context
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/util/compat.py", line 128, in reraise
    raise value.with_traceback(tb)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages/sqlalchemy/engine/default.py", line 550, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column "ti
tle" does not exist
LINE 1: ...,"year","runtime","imdbID","imdbRating") VALUES ((title), (y...
                                                             ^
HINT:  There is a column named "title" in table "movies", but it cannot be re
ferenced from this part of the query.

[SQL: INSERT INTO movies("title","year","runtime","imdbID","imdbRating") VALU
ES ((title), (year), (runtime), (imdbID), (imdbRating))]
[parameters: {'title': 'Title', 'year': 'Year', 'runtime': 'Runtime', 'imdbID
': 'imdbID', 'imdbRating': 'imdbRating\n'}]
(Background on this error at: http://sqlalche.me/e/f405)

Simon King

unread,
Jun 13, 2019, 7:10:41 AM6/13/19
to sqlal...@googlegroups.com
Your "engine.execute" statement looks wrong to me:

engine.execute('INSERT INTO
movies("title","year","runtime","imdbID","imdbRating") VALUES
((title), (year), (runtime), (imdbID), (imdbRating))',
{"title": title, "year": year, "runtime": runtime, "imdbID":
imdbID, "imdbRating": imdbRating })

In particular, the VALUES clause is wrong. That's not the way that you
specify parameters when executing a parametrized query, so the
database is interpreting "(title)" as a reference to a column, rather
than a reference to a parameter.

You're probably best off using SQLAlchemy's text() construct:

https://docs.sqlalchemy.org/en/13/core/tutorial.html#using-textual-sql

You would put this outside the loop:

insert_statement = sqlalchemy.text(
"INSERT INTO movies(title, year, runtime, imdbID, imdbRating)
VALUES (:title, :year, :runtime, :imdbID, :imdbRating)"
)

and then inside the loop you would use:

engine.execute(insert_statement, title=title, year=year,
runtime=runtime, imdbID=imdbID, imdbRating=imdbRating)

Also, I suspect (but haven't verified) that "engine.execute()" will
check out a connection from the pool, run the statement, and commit a
transaction. If you want to import all the rows in a single
transaction, you should probably explicitly create a connection at the
beginning, use "connection.execute()" to run your SQL, and commit it
at the end. See
https://docs.sqlalchemy.org/en/13/core/connections.html#using-transactions
for examples.

Hope that helps,

Simon
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.
> ---
> You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.
> To post to this group, send email to sqlal...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/ca1b1197-3f05-4572-a11a-fda4e08d27f3%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Cravan

unread,
Jun 13, 2019, 7:20:50 AM6/13/19
to sqlal...@googlegroups.com
I got a new error after using Simon's suggestion, would someone help me?
######################################
File "import.py", line 24
insert_statement = sqlalchemy.text("INSERT INTO movies(title, year,
runtime, "imdbID", "imdbRating") VALUES (:title, :year, :runtime, :imd
bID, :imdbRating)")

^
SyntaxError: invalid syntax
Thanks,
Cravan
To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/CAFHwexcqMjc_vUtNhx-dpyS6ECK5TSDeObDEGzeOFR1n35JkdQ%40mail.gmail.com.
import.py
summative3.sql

Simon King

unread,
Jun 13, 2019, 7:36:25 AM6/13/19
to sqlal...@googlegroups.com
You've got double-quotes around "imdbID" and "imdbRating" on that
line, and double-quotes around the SQL statement, which is a syntax
error. I think you should be able to just remove the quotes around the
column names, but if you really need them for some reason, replace the
outer double-quotes with single-quotes.

Simon
> To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/2E8C06E7-BD2D-408A-BFE4-6657C5270CF5%40gmail.com.

Cravan

unread,
Jun 13, 2019, 7:40:12 AM6/13/19
to sqlal...@googlegroups.com
Alright, thanks because I created my sql column names with imdbID, which contains caps.
However, after changing it to single-quotes, I got a syntax error.
###################################
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_conte
xt
cursor, statement, parameters, context
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/engine/default.py", line 550, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.SyntaxError: syntax error at or near "'imdbID'"
LINE 1: INSERT INTO movies(title, year, runtime, 'imdbID', 'imdbRati...
^


The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "import.py", line 28, in <module>
main()
File "import.py", line 25, in main
engine.execute(insert_statement, title=title, year=year, runtime=ru
ntime, imdbID=imdbID, imdbRating=imdbRating)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/engine/base.py", line 2166, in execute
return connection.execute(statement, *multiparams, **params)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_co
nnection
return connection._execute_clauseelement(self, multiparams, params)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_claus
eelement
distilled_params,
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_conte
xt
e, statement, parameters, cursor, context
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_
exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_caus
e
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_conte
xt
cursor, statement, parameters, context
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/site-packages/sqlalchemy/engine/default.py", line 550, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.SyntaxError) syntax e
rror at or near "'imdbID'"
LINE 1: INSERT INTO movies(title, year, runtime, 'imdbID', 'imdbRati...
[SQL: INSERT INTO movies(title, year, runtime, 'imdbID', 'imdbRating')
VALUES (%(title)s, %(year)s, %(runtime)s, %(imdbID)s, %(imdbRating)s)]
[parameters: {'title': 'The Lego Movie', 'year': 2014, 'runtime': 100,
'imdbID': 'tt1490017', 'imdbRating': 7.8}]
(Background on this error at: http://sqlalche.me/e/f405)
Thanks,
Cravan
To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/CAFHwexexCE%3DpS-wP6Nxbd2znhesjwtuzt1zjodPytdiKF-x7uQ%40mail.gmail.com.
import.py
summative3.sql

Simon King

unread,
Jun 13, 2019, 8:19:52 AM6/13/19
to sqlal...@googlegroups.com
It's worth pointing out that your first and last errors are database
errors from postgresql, whereas the middle syntax error was a Python
error.

In Python, single and double-quotes are interchangeable, and you can
use whichever is more convenient for you. This is not true for
postgresql. In PG, single quotes are used for strings, and
double-quotes are used for identifiers that you want to be treated
literally (eg. because they contain characters that aren't normally
allowed in identifiers, or because you want them to be
case-sensitive).

So in this case, you need imdbID and imdbRating to be surrounded by
double-quotes because you want them to be handled as case-sensitive.
The easiest thing to do is to change the quotes around the whole SQL
statement to be single quotes.

ie.

sqlalchemy.text('INSERT INTO movies(title, year, runtime, "imdbID",
"imdbRating") VALUES(...)')

Simon
> To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/4E9096F6-7FD9-4AAF-A045-0DBCE352FE9F%40gmail.com.

Cravan

unread,
Jun 13, 2019, 8:26:37 AM6/13/19
to sqlal...@googlegroups.com
Thanks Simon, I got a new error this time:
#####################
Traceback (most recent call last):
File "import.py", line 28, in <module>
main()
File "import.py", line 20, in main
year = int(row[1])
IndexError: list index out of range

Thanks,
Cravan
To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeK05VEPLk1cLH2iq0tSzpunWViNd2DRCDLZdQi_H85xA%40mail.gmail.com.
import.py
movies.csv
summative3.sql

Simon King

unread,
Jun 13, 2019, 8:53:19 AM6/13/19
to sqlal...@googlegroups.com
You have an empty row in your CSV file (maybe a blank line at the end?).

Try adding this at the beginning of your loop:

if not row:
continue
> To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/ECE19F5A-993C-4063-957F-C198A1591A13%40gmail.com.

Cravan

unread,
Jun 13, 2019, 9:21:48 AM6/13/19
to sqlal...@googlegroups.com
Hi,
I got the same error even after adding that.
#########################
Traceback (most recent call last):
File "import.py", line 30, in <module>
main()
File "import.py", line 22, in main
year = int(row[1])
IndexError: list index out of range
Thanks,
Cravan
To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/CAFHwexeAZ4SYJGD_zmh0NV97RtMER1Cq5Cr7FM%3D-5TeAAX1_vQ%40mail.gmail.com.
summative3.sql
import.py
movies.csv

Simon King

unread,
Jun 13, 2019, 10:27:33 AM6/13/19
to sqlal...@googlegroups.com
In that case I guess your "row" list only has 1 element. Try putting
"print(i, row)" before the "title = row[0]" line.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/A8B78051-BE1B-4D1B-BEAB-01F595138DE3%40gmail.com.

Cravan

unread,
Jun 13, 2019, 10:39:00 AM6/13/19
to sqlal...@googlegroups.com
Hi Simon,
Here's the result, although I actually have 250, it stopped at 144. I have attached the codes and the .csv sheet for your reference.


1 ['The Lego Movie', '2014', '100', 'tt1490017', '7.8']
2 ['The Shawshank Redemption', '1994', '142', 'tt0111161', '9.3']
3 ['The Godfather', '1972', '175', 'tt0068646', '9.2']
4 ['The Godfather: Part II', '1974', '202', 'tt0071562', '9']
5 ['The Dark Knight', '2008', '152', 'tt0468569', '9']
6 ['12 Angry Men', '1957', '96', 'tt0050083', '8.9']
7 ["Schindler's List", '1993', '195', 'tt0108052', '8.9']
8 ['The Lord of the Rings: The Return of the King', '2003', '201', '
tt0167260', '8.9']
9 ['Pulp Fiction', '1994', '154', 'tt0110912', '8.9']
10 ['The Good the Bad and the Ugly', '1966', '178', 'tt0060196', '8.
9']
11 ['Fight Club', '1999', '139', 'tt0137523', '8.8']
12 ['The Lord of the Rings: The Fellowship of the Ring', '2001', '17
8', 'tt0120737', '8.8']
13 ['Forrest Gump', '1994', '142', 'tt0109830', '8.8']
14 ['Star Wars: Episode V - The Empire Strikes Back', '1980', '124',
'tt0080684', '8.7']
15 ['Inception', '2010', '148', 'tt1375666', '8.8']
16 ['The Lord of the Rings: The Two Towers', '2002', '179', 'tt0167261', '8.7']
17 ['Avengers: Endgame', '2019', '181', 'tt4154796', '8.9']
18 ["One Flew Over the Cuckoo's Nest", '1975', '133', 'tt0073486', '8.7']
19 ['Goodfellas', '1990', '146', 'tt0099685', '8.7']
20 ['The Matrix', '1999', '136', 'tt0133093', '8.7']
21 ['Seven Samurai', '1954', '207', 'tt0047478', '8.7']
22 ['Se7en', '1995', '127', 'tt0114369', '8.6']
23 ['City of God', '2002', '130', 'tt0317248', '8.6']
24 ['Star Wars: Episode IV - A New Hope', '1977', '121', 'tt0076759', '8.6']
25 ['The Silence of the Lambs', '1991', '118', 'tt0102926', '8.6']
26 ["It's a Wonderful Life", '1946', '130', 'tt0038650', '8.6']
27 ['Life Is Beautiful', '1997', '116', 'tt0118799', '8.6']
28 ['Spirited Away', '2001', '125', 'tt0245429', '8.6']
29 ['Saving Private Ryan', '1998', '169', 'tt0120815', '8.6']
30 ['The Usual Suspects', '1995', '106', 'tt0114814', '8.6']
31 ['Leon: The Professional', '1994', '110', 'tt0110413', '8.6']
32 ['The Green Mile', '1999', '189', 'tt0120689', '8.6']
33 ['Interstellar', '2014', '169', 'tt0816692', '8.6']
34 ['Psycho', '1960', '109', 'tt0054215', '8.5']
35 ['American History X', '1998', '119', 'tt0120586', '8.5']
36 ['City Lights', '1931', '87', 'tt0021749', '8.5']
37 ['Casablanca', '1942', '102', 'tt0034583', '8.5']
38 ['Once Upon a Time in the West', '1968', '165', 'tt0064116', '8.5']
39 ['The Pianist', '2002', '150', 'tt0253474', '8.5']
40 ['Modern Times', '1936', '87', 'tt0027977', '8.5']
41 ['The Intouchables', '2011', '112', 'tt1675434', '8.5']
42 ['The Departed', '2006', '151', 'tt0407887', '8.5']
43 ['Back to the Future', '1985', '116', 'tt0088763', '8.5']
44 ['Terminator 2: Judgment Day', '1991', '137', 'tt0103064', '8.5']
45 ['Whiplash', '2014', '106', 'tt2582802', '8.5']
46 ['The Lion King', '1994', '88', 'tt0110357', '8.5']
47 ['Rear Window', '1954', '112', 'tt0047396', '8.5']
48 ['Gladiator', '2000', '155', 'tt0172495', '8.5']
49 ['Raiders of the Lost Ark', '1981', '115', 'tt0082971', '8.5']
50 ['The Prestige', '2006', '130', 'tt0482571', '8.5']
51 ['Apocalypse Now', '1979', '147', 'tt0078788', '8.5']
52 ['Memento', '2000', '113', 'tt0209144', '8.5']
53 ['Alien', '1979', '117', 'tt0078748', '8.5']
54 ['Grave of the Fireflies', '1988', '89', 'tt0095327', '8.5']
55 ['Cinema Paradiso', '1988', '155', 'tt0095765', '8.5']
56 ['Spider-Man: Into the Spider-Verse', '2018', '117', 'tt4633694', '8.5']
57 ['The Great Dictator', '1940', '125', 'tt0032553', '8.5']
58 ['Sunset Boulevard', '1950', '110', 'tt0043014', '8.4']
59 ['The Lives of Others', '2006', '137', 'tt0405094', '8.4']
60 ['Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb', '1964', '95', '
tt0057012', '8.4']
61 ['Avengers: Infinity War', '2018', '149', 'tt4154756', '8.5']
62 ['Paths of Glory', '1957', '88', 'tt0050825', '8.4']
63 ['Django Unchained', '2012', '165', 'tt1853728', '8.4']
64 ['The Shining', '1980', '146', 'tt0081505', '8.4']
65 ['WALL-E', '2008', '98', 'tt0910970', '8.4']
66 ['Princess Mononoke', '1997', '134', 'tt0119698', '8.4']
67 ['Witness for the Prosecution', '1957', '116', 'tt0051201', '8.4']
68 ['Oldboy', '2003', '120', 'tt0364569', '8.4']
69 ['Aliens', '1986', '137', 'tt0090605', '8.4']
70 ['The Dark Knight Rises', '2012', '164', 'tt1345836', '8.4']
71 ['American Beauty', '1999', '122', 'tt0169547', '8.4']
72 ['Once Upon a Time in America', '1984', '229', 'tt0087843', '8.4']
73 ['Coco', '2017', '105', 'tt2380307', '8.4']
74 ['Das Boot', '1981', '149', 'tt0082096', '8.4']
75 ['Citizen Kane', '1941', '119', 'tt0033467', '8.3']
76 ['Braveheart', '1995', '178', 'tt0112573', '8.4']
77 ['Vertigo', '1958', '128', 'tt0052357', '8.3']
78 ['North by Northwest', '1959', '136', 'tt0053125', '8.3']
79 ['Reservoir Dogs', '1992', '99', 'tt0105236', '8.3']
80 ['Your Name', '2015', '22', 'tt6033368', '8.9']
81 ['Star Wars: Episode VI - Return of the Jedi', '1983', '131', 'tt0086190', '8.3']
82 ['M', '1931', '99', 'tt0022100', '8.3']
83 ['Amadeus', '1984', '160', 'tt0086879', '8.3']
84 ['Requiem for a Dream', '2000', '102', 'tt0180093', '8.3']
85 ['Dangal', '2016', '161', 'tt5074352', '8.5']
86 ['3 Idiots', '2009', '170', 'tt1187043', '8.4']
87 ['2001: A Space Odyssey', '1968', '149', 'tt0062622', '8.3']
88 ['Toy Story', '1995', '81', 'tt0114709', '8.3']
89 ['Like Stars on Earth', '2007', '165', 'tt0986264', '8.4']
90 ['Eternal Sunshine of the Spotless Mind', '2004', '108', 'tt0338013', '8.3']
91 ['Lawrence of Arabia', '1962', '216', 'tt0056172', '8.3']
92 ['A Clockwork Orange', '1971', '136', 'tt0066921', '8.3']
93 ["Singin' in the Rain", '1952', '103', 'tt0045152', '8.3']
94 ['Amelie', '2001', '122', 'tt0211915', '8.3']
95 ['Double Indemnity', '1944', '107', 'tt0036775', '8.3']
96 ['Inglourious Basterds', '2009', '153', 'tt0361748', '8.3']
97 ['Taxi Driver', '1976', '114', 'tt0075314', '8.3']
98 ['Full Metal Jacket', '1987', '116', 'tt0093058', '8.3']
99 ['To Kill a Mockingbird', '1962', '129', 'tt0056592', '8.3']
100 ['Bicycle Thieves', '1948', '89', 'tt0040522', '8.3']
101 ['Good Will Hunting', '1997', '126', 'tt0119217', '8.3']
102 ['The Kid', '1921', '68', 'tt0012349', '8.3']
103 ['The Sting', '1973', '129', 'tt0070735', '8.3']
104 ['The Hunt', '2012', '115', 'tt2106476', '8.3']
105 ['Toy Story 3', '2010', '103', 'tt0435761', '8.3']
106 ['Snatch', '2000', '102', 'tt0208092', '8.3']
107 ['Scarface', '1983', '170', 'tt0086250', '8.3']
108 ['For a Few Dollars More', '1965', '132', 'tt0059578', '8.3']
109 ['The Apartment', '1960', '125', 'tt0053604', '8.3']
110 ['Metropolis', '1927', '153', 'tt0017136', '8.3']
111 ['Monty Python and the Holy Grail', '1975', '91', 'tt0071853', '8.3']
112 ['L.A. Confidential', '1997', '138', 'tt0119488', '8.3']
113 ['A Separation', '2011', '123', 'tt1832382', '8.3']
114 ['Indiana Jones and the Last Crusade', '1989', '127', 'tt0097576', '8.2']
115 ['Up', '2009', '96', 'tt1049413', '8.2']
116 ['Rashomon', '1950', '88', 'tt0042876', '8.3']
117 ['All About Eve', '1950', '138', 'tt0042192', '8.3']
118 ['Batman Begins', '2005', '140', 'tt0372784', '8.2']
119 ['Some Like It Hot', '1959', '121', 'tt0053291', '8.2']
120 ['Yojimbo', '1961', '110', 'tt0055630', '8.3']
121 ['Downfall', '2004', '156', 'tt0363163', '8.2']
122 ['Unforgiven', '1992', '130', 'tt0105695', '8.2']
123 ['Die Hard', '1988', '132', 'tt0095016', '8.2']
124 ['Heat', '1995', '170', 'tt0113277', '8.2']
125 ['The Treasure of the Sierra Madre', '1948', '126', 'tt0040897', '8.2']
126 ['Incendies', '2010', '131', 'tt1255953', '8.3']
127 ['Ikiru', '1952', '143', 'tt0044741', '8.3']
128 ['Green Book', '2018', '130', 'tt6966692', '8.3']
129 ['Raging Bull', '1980', '129', 'tt0081398', '8.2']
130 ['The Great Escape', '1963', '172', 'tt0057115', '8.2']
131 ['Children of Heaven', '1997', '89', 'tt0118849', '8.3']
132 ["Pan's Labyrinth", '2006', '118', 'tt0457430', '8.2']
133 ['Chinatown', '1974', '130', 'tt0071315', '8.2']
134 ['My Neighbor Totoro', '1988', '86', 'tt0096283', '8.2']
135 ['The Third Man', '1949', '93', 'tt0041959', '8.2']
136 ["Howl's Moving Castle", '2004', '119', 'tt0347149', '8.2']
137 ['My Father and My Son', '2005', '108', 'tt0476735', '8.4']
138 ['Ran', '1985', '162', 'tt0089881', '8.2']
139 ['Judgment at Nuremberg', '1961', '186', 'tt0055031', '8.2']
140 ['The Secret in Their Eyes', '2009', '129', 'tt1305806', '8.2']
141 ['The Gold Rush', '1925', '95', 'tt0015864', '8.2']
142 ['A Beautiful Mind', '2001', '135', 'tt0268978', '8.2']
143 ['The Bridge on the River Kwai', '1957', '161', 'tt0050212', '8.2']
144 ['Lock, Stock and Two Smoking Barrels;1998;107;tt0120735;8.2']
#################################
Traceback (most recent call last):
File "import.py", line 31, in <module>
main()
File "import.py", line 23, in main
year = int(row[1])
IndexError: list index out of range
Cravan:cepsummative3 CPZ$


Thanks,
Cravan
To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/CAFHwexfijvidk5PyFJkS2-XO0%3DX3zxP32QKOQULNiso-iHcfUQ%40mail.gmail.com.
movies.csv
import.py
summative3.sql

Simon King

unread,
Jun 13, 2019, 10:49:53 AM6/13/19
to sqlal...@googlegroups.com
This line is normal (the list contains 5 strings):

143 ['The Bridge on the River Kwai', '1957', '161', 'tt0050212', '8.2']

This line is not normal (the list only contains a single string):

144 ['Lock, Stock and Two Smoking Barrels;1998;107;tt0120735;8.2']

You should look at your CSV file (in a text editor, not a spreadsheet
application) and see what is different about the "Lock, Stock" line
compared to the "River Kwai" line.

Simon
> To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/588242CF-3AA5-43D5-8C20-9D384995E7F6%40gmail.com.

Cravan

unread,
Jun 13, 2019, 11:01:20 AM6/13/19
to sqlal...@googlegroups.com
Dear Simon,
You're right! Thanks, I realised there was a pair of double quotations. However, I received another error related to the commit function:
################################
Traceback (most recent call last):
File "import.py", line 31, in <module>
main()
File "import.py", line 29, in main
engine.commit() # transactions are assumed, so close the transaction finished
AttributeError: 'Engine' object has no attribute 'commit'
Thanks,
Cravan
To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/CAFHwexdbjhLsGFdf29fONzpos53On0uUbA3J6ZAoJ7TU0if4CQ%40mail.gmail.com.

Simon King

unread,
Jun 13, 2019, 11:41:24 AM6/13/19
to sqlal...@googlegroups.com
The engine manages a pool of connections. When you call
engine.execute(), you are running that single statement in its own
transaction. That's why engines don't have a commit() method.

Instead, you should create an explicit connection at the beginning of
the function, use connection.execute() rather than engine.execute(),
and call connection.commit() at the end:

https://docs.sqlalchemy.org/en/13/core/connections.html#using-transactions

Simon
> To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/09372F2C-3BB3-4931-BDE8-2DA7EDCF4A95%40gmail.com.

Mike Bayer

unread,
Jun 13, 2019, 1:53:16 PM6/13/19
to sqlal...@googlegroups.com
this is just the reason I'm hoping to greatly reduce the choices available with engines/connections by the time we get to SQLAlchemy 2.0 (eg. no more connectionless execution or autocommit)
Reply all
Reply to author
Forward
0 new messages