Scraped Data on NYC's 271,000 Employees

[Chris Whong]

Hi Devs,

While stuck on trains this weekend, I put together a node scraper for the Position Schedules located in the NYC Budget. (spun-off from some earlier work on budgetBuddy) A screenshot of a raw position schedule is attached.

The position schedules include a row for each Full-time job in the city, along with an annual salary range, number of positions, and total spend.  For example, the attached schedule includes a line called "Teacher", which has 47,434 positions. The total annual spend for all of these Teacher positions is $3.6 Billion, averaging to $75,269.47 annual salary per teacher.  

Fun fact:  According to the data, there are 271,589 full-time positions authorized in the budget, most of whom are teachers and police officers.

The output data is here.  

I munged this data for a visualization project, but wanted to share it in case anyone else can think of an interesting way to visualize/analyze it. 

-Chris Whong

[Steven Romalewski]

Chris,

In case you don't know about SeeThroughNY, they've FOILed this data and more, and have posted it on searchable format here: http://seethroughny.net/city-of-ny

Not sure if this is exactly what you've scraped, or if it's as current, but it's worth comparing with your effort.

Steve

[Ralph Yozzo]

Hi Steve and Everyone,

Interesting, 

Does anyone know if there are errors in the data.

For example,

Glenn Newman Tax Commission President is paid about $200K

But the default search shows a DEP person making $700K  is that correct?  this person is paid more than the mayor?

is this yearly? or lifetime?  also does it include pension or awards or something?

See

$770000 / $421 / day

1828 days  

This would mean that Gerald Mistretta somehow was able to work 1828 days in 2011  that would be a fair amount of overtime.

Even if Gerald worked every single day during the entire year he would have to been working at 5 times pay OT mode :)

Amazing!

--
This is the BetaNYC Developers list. You can find projects at < projects.betanyc.us > or ideas at < betaNYC.ideascale.com >.

BetaNYC is committed to hosting safe and open spaces for all. By participating in this space you are committing yourself to BetaNYC's Code of Conduct and Anti-Harassment Policy. < bit.ly/betanyc-coc >
---
You received this message because you are subscribed to the Google Groups "BetaNYC-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to betanyc-dev...@googlegroups.com.
To post to this group, send email to betan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/betanyc-dev/67b33cf0-cf5b-4e27-b581-d00c78e3e3f5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Chris Whong]

Thanks for sharing Steve, I had come across that site a while back while working on govSee (a web org-chart for governments, now defunct) but learned that their data is not current. I never figured out why, does the city just not have to share salary data for a certain amount of time?  

Just like with everything else in the budget, I would love to compare the actuals versus the budget amounts and see how things marry up.  

-Chris Whong

[Ralph Yozzo]

Hi Chris

How do you know the data is not current?

The query for Glenn newman shows his 2014 pay.  

Also I question some of the extremely large salaries.   Does anyone know how a dep worker could be paid $700K???

To view this discussion on the web visit https://groups.google.com/d/msgid/betanyc-dev/CA%2BOZq%2Ba_uJ-9UXH5JGz4RxbWSrNdLu1RjiX0tQbVzxhjOS_OMw%40mail.gmail.com.

[Chris Whong]

Oh, I think I was looking in the wrong place... I didn't really look around this time since I'm on vacation but I recall there only being old data when I looked a few years ago.

-C

To view this discussion on the web visit https://groups.google.com/d/msgid/betanyc-dev/CAGDR%3DQ0VHcNCkUnN_GavDsvcns63%2BEXAZ6QYBc_ZjUuS4wrbrA%40mail.gmail.com.

[Steven Romalewski]

Ralph, I can't answer definitive re the seemingly high salaries. But I wonder if the $700k figure is just an annualized version of the daily rate. So, the employee doesn't really make that much - maybe they just work two days a week for 3 months, hypothetically - but since SeeThroughNY shows all the salaries as annual, they convert the daily rates to annual just to be consistent?

Steve

[Adam Becker]

I mean, that would come out to $2,800/day -- seems that either the data is bad (or includes awards, or some other unknown factor), or there's some serious Bad Stuff (tm) going on.

So... how does one proceed, besides finding someone at Gothamist to write about this? 

On Wed, Oct 15, 2014 at 10:17 PM, Steven Romalewski <sroma...@gc.cuny.edu> wrote:

Ralph, I can't answer definitive re the seemingly high salaries. But I wonder if the $700k figure is just an annualized version of the daily rate. So, the employee doesn't really make that much - maybe they just work two days a week for 3 months, hypothetically - but since SeeThroughNY shows all the salaries as annual, they convert the daily rates to annual just to be consistent?

Steve

--
This is the BetaNYC Developers list. You can find projects at < projects.betanyc.us > or ideas at < betaNYC.ideascale.com >.

BetaNYC is committed to hosting safe and open spaces for all. By participating in this space you are committing yourself to BetaNYC's Code of Conduct and Anti-Harassment Policy. < bit.ly/betanyc-coc >
---
You received this message because you are subscribed to the Google Groups "BetaNYC-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to betanyc-dev...@googlegroups.com.
To post to this group, send email to betan...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/betanyc-dev/bb212d18-963e-4b1b-b819-96ddee01fee4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Adam Becker

510.928.9111

@AdamJacobBecker

[Ralph Yozzo]

Hi Adam,

I agree!  but for this particular over $700K case for DEP workers it appears to a collective bargaining agreement for retroactive payments.  See

(2) In the 2011 data, there are 877 daily employees of the Department of Environmental Protection who earned over $200,000 who received retroactive payments for a collective bargaining agreement that was settled in 2010.

I cannot see to find a reference for this particular 2010 bargaining agreement

But if you search for "retroactive pay collective bargaining nyc" you will find many references.

It is no wonder the budget/tax/revenue/etc is such an opaque process with retroactive deals, corruption, member items, discretionary funds, etc. etc.

Imagine, an employee comes to you saying "he/she needs $500K in retroactive pay" or you say to your employer, "I need to be paid $500K for work I did many years ago."

To view this discussion on the web visit https://groups.google.com/d/msgid/betanyc-dev/CAJDyKVO8GdN0OQHPGJvkrQ_ejR-50Z8J4cdbZRpRMzQJtLi-9w%40mail.gmail.com.