tango.info festival data

1 view
Skip to first unread message

dekaysion

unread,
Jul 6, 2007, 1:42:21 PM7/6/07
to Tango 2.0
Alright, I am just sitting here working on my first ruby project ever.
Fun, to say the least.

Since Tobias just mentioned he has his festivals added to his wiki
(e.g. at http://wiki.tango.info/mul/Category:festivals) I thought it
would be fun to just parse all events in 2007 and create a calendar
from it. Easy, right?

Well... I am still in the process of doing so ;) Just wanted to express
my wish for a direct link to Tobias's festival link from the wiki
page, since that seems to be missing...

...will keep you posted how far I get.

dekaysion

unread,
Jul 6, 2007, 5:45:27 PM7/6/07
to Tango 2.0
not completely done, but finished for today:

require "rubygems"
require "mechanize"
require 'icalendar'
require 'date'
include Icalendar # Probably do this in your class to limit namespace
overlap


class TangoInfoExtractor < WWW::Mechanize
def process

cal = Calendar.new

user_agent_alias = 'Mac FireFox'
get "http://wiki.tango.info/mul/Category:festivals"

# page.links.each do |link|
# puts link.text
# end

page.links.with.href( %r{2007-12} ).each do |link|

puts link.href
begin
transact do
test = get link
content = (test/"#bodyContent")
festivalURL = (content/"p/a").inner_html
p festivalURL
festivalDescription = (content/"ul")

event = Event.new

s = link.text.scan(/[\d-]+/)
t = s[0].scan(/\d+/)
# if we only have 3 numbers it is a one day event
if t[3] == nil
event.start = DateTime.civil(t[0].to_i, t[1].to_i,
t[2].to_i)
event.end = DateTime.civil(t[0].to_i, t[1].to_i,
t[2].to_i)

# if we have 4 numbers it is a multi day event, but within
one month

elsif t[4] == nil
event.start = DateTime.civil(t[0].to_i, t[1].to_i,
t[2].to_i)
event.end = DateTime.civil(t[0].to_i, t[1].to_i,
t[3].to_i)

# if we have 5 numbers we have a multi day event spanning
multiple months
else
event.start = DateTime.civil(t[0].to_i, t[1].to_i,
t[2].to_i)
event.end = DateTime.civil(t[0].to_i, t[3].to_i,
t[4].to_i)

# don't worry about spanning years for now.
end

event.categories = ["tango", "festival"]

# Now with support for property parameters
event.url = festivalURL
event.summary = link.text
event.description = festivalDescription.inner_html
cal.add_event(event)
end
# Now we're back at the original page.
rescue => e
$stderr.puts "#{e.class}: #{e.message}"
end

end

# We can output the calendar as a string to write to a file,
# network port, database etc.
cal_string = cal.to_ical
puts cal_string
end
end

TangoInfoExtractor.new.process

PeterF

unread,
Jul 8, 2007, 2:01:13 PM7/8/07
to Tango 2.0
do you have a demo running somewhere?

dekaysion

unread,
Jul 9, 2007, 9:19:09 AM7/9/07
to Tango 2.0
Yes, on my home machine. Too bad it is not accessible on the web. I
believe I have some hosting place that might run ruby, but I will have
to check at home (though I doubt the hoster has mechanize).

You could probably run it via a cron job?

tob...@tango.info

unread,
Jul 9, 2007, 2:45:20 PM7/9/07
to tang...@googlegroups.com
For Berlin I started putting events to individual locations.

see the subcategories at:
http://wiki.tango.info/mul/Category:DEBER

I think: Date in page title, Location via category.

I have to think about a solution for repeating events. Before I have
no good idea for that I am unwilling to programm ics. I am just not
sure whether the wiki is the right thing.


--
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin, Germany

http://info.tango.info http://eng.tango.info
http://festivals.tango.info http://videos.tango.info
http://wiki.tango.info

Reply all
Reply to author
Forward
0 new messages