append() takes no string arguments

14 views
Skip to first unread message

Shriman Sonti

unread,
Dec 22, 2016, 7:52:15 PM12/22/16
to PyData
Hello guys i have another question
So I want to put multiple csv files together and were running into sort of a problem.
So the code:
    import glob
    import pandas as pd

# get data file names
path =r'C:\Users\spkso\Desktop\in'
filenames = glob.glob(path + "/*.csv")

dfs = []
for filename in filenames:
    print(filename)
    csv=pd.read_csv(filename)
    dfs.append(csv)
print(dfs)


returns this:

C:\Users\spkso\Desktop\in\qbr2014.csv
C:\Users\spkso\Desktop\in\qbr2015.csv
C:\Users\spkso\Desktop\in\qbr2016.csv
[            QB  Rating
0  Andrew Luck    96.5
1   Drew Brees    92.4
2    Tom Brady    95.2,             QB   Rating
0  Andrew Luck     90.4
1   Drew Brees     95.6
2    Tom Brady     99.8,             QB  Rating
0  Andrew Luck   101.2
1   Drew Brees    94.6
2    Tom Brady   100.3]


The only problem i am having with this code is the fact that there are two headers QB Rating outside of the list

Do you guys have any suggestions 
It would be much appreciated
    

Paul Hobson

unread,
Dec 23, 2016, 11:39:18 AM12/23/16
to pyd...@googlegroups.com
You're appending to a list, not a single dataframe. What you're doing is the same thing as this:

mylist = [] 
mylist.append('cat')
mylist.append('dog')
mylist.append('puppy')
mylist.append([1, 2, 3])

No matter what we append to it, it's still a list.

To get a single dataframe out of your list, you need to use the pandas.concat function:

dfs = []
for filename in filenames:
    print(filename)
    csv = pandas.read_csv(filename)
    dfs.append(csv)
my_df = pandas.concat(dfs)
print(my_df)

A pattern that I use a lot is to place the for-loop into a list comprehension:

my_df = pandas.concat([pandas.read_csv(fn) for fn in filenames])
Reply all
Reply to author
Forward
0 new messages