intern log: sharing the cool things you see at your intern

[Anil]

Mostly when we build stuff (hacks), we never consider scalability, testing, etc. It is just a thing tested on our browser / device. If its works it gets out. Many of us do get to intern at some of big tech firms and get insight into working there. This is thread is for sharing your experience. Just talk about what you working on, systems and tools used and general work culture.

Myself interning in the Networking group at MSR. 

Cant exactly talk of my work but it revolves around the computations in big data clusters. You are used to SQL, Hadoop but people here have their own distributed system (Dryad) and a complete extendable query language (SCOPE) which extends SQL with C#. 

Big Data: 

Talk of jobs, some queries which I run have 5+ levels of nesting with custom processor/reducer and process TB's of data - 20,000 nodes, answer out in minutes. Seems just amazing the tools internally they have at their disposable - very hard for a startup to stand when they enter the market segment. 

Coding:

A lot of research is applied meaning it has practical relevance. Have to work on windows which was frustrating at start but somehow just end up loving all the tools they have built for Visual Studio. It is C# .NET everywhere. Also now use Bing + IE (in office)!

Dunno about software development teams but at-least MSR houses some really smart people. They claim to have the highest number of PhD's(CS) in one building. Fun loving no nonsense type of people. In comparison to other other intern accounts where travel is more exciting here work feels more exciting. 

[Ashwin Paranjape]

I'm too am interning at MSR and my job is to design learning algorithms for something which I won't tell.

Design and Experimentation: Theory differs hugely from practice especially where if someone is trying to learn from real data. Case in Point - I had to design a set of experiments to understand how some parameters affect outcomes and then tune them. An algorithm that is guaranteed to converge quadratically may turn out slower in practice than something which is not and the best way to know that is to experiment.

Execution Speed: Until now I had never thought about making some code run faster. We only learn reducing order of algorithms. But that really is a misnomer. Order of an algorithm is measured wrt to input size, but what if it doesn't fit in cache or even RAM? To make an algorithm run as fast as predicted in theory is a completely different ball game. This is even more important when large datasets are considered and you have to do many iterations to tune it. But its not always about powerful hardware. Even in our laptops there are 2-4 cores and support for SSE instructions. And how important is execution speed? In a talk someone (from outside Microsoft) said they were not deploying a certain speech recognition service because it takes 5 ms whereas it should take 2 ms to process.

Exposure: So many people are working on so many different areas. It makes sense to keep track of what they are doing because someone might have faced a similar problem and found out a solution. No point in reinventing the wheel!

--
--
The website for the club is http://stab-iitb.org/wncc
To post to this group, send email to wncc...@googlegroups.com
 
---
You received this message because you are subscribed to the Google Groups "Web and Coding Club IIT Bombay" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wncc_iitb+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Ashwin P. Paranajape

Manager, Technovation

Visiting Scientist IST Austria

3rd year CSE Btech.
IIT Bombay

[Saswat Padhi]

Interesting stuff Anil; and Ashwin!

I work with Google's infrastructure team. To be specific; I work with the Borg-Omega team.
I am not allowed to share confidential information but you can find lots of stuff (there's a nice talk on YouTube as well) about both Borg and Omega. Very crudely, Borg can be called as the OS that runs on all data centers in Google. You submit jobs and Borg takes care of running an monitoring the jobs. Google is scaling up it's infrastructure. And for scaling up at "Google" level, pretty much the entire infrastructure framework is re-designed and rewritten. Omega is the successor to Borg.

The best part about working with Google is the rich set of dev-tools they have. Every process: compilation, linking, execution, testing is completely distributed. There are lots of amazing proprietary tools and concepts in Google that makes development really fast and hassle free. Plus, the documentation of the code base is extremely good! The internal style guides and strict code reviews stress a lot on writing "readable" code. Although I myself haven't found time for a 20% project; it's interesting to browse through other people's projects! There are lots of brilliant projects waiting in queue and the 80/20 rule is a great initiative by Google.

I work on things related to both Borg and Omega and bridging them. Being in the infrastructure team gives me direct access to play with some of the data centers. I develop mostly in C++ and Python; and use tons of cool technologies. Working with some of the smartest people and getting your hands on some of the coolest products right now, is definitely something that I wouldn't expect anywhere else! And every day at work is so exciting and so refreshing, you start looking forward to Mondays on Saturdays!

Needless to say .. the Google office is not just for "work". With XBox, bowling alleys, pool/TT/Foosball tables, nap pods, massage centers, parks and a lake nearby; it's a perfect place to chill out as well :-)

Saswat Padhi  |

Software Engineering Intern |

#Borg #Omega Testing |

Google Infrastructure

[Saif Hasan]

This is really nice stuff. Waiting for more responses. 

Great Initiative Anil.

[Kanishk Parihar]

+1 to Ashwin on the execution speed point and working with data that cant fit in memory.

Hey all. I am interning at facebook in the infrastructure genre. To be precise, I work on memcache at facebook.

Facebook memcaches handles a humungous amount of data per unit time somewhere around 1 billion requests per second. My work goes into making the facebook, faster, reliable and secure.

Exposure: Working in big ass tech companies is really exciting, as you are exposed to a level you cant get in IITB. The issues that arise when you are working on distributed systems, is quite something!

The scale of code that is written is high, and the components the code integrates with are distributed over networks and machines. Basically, everything on servers is distributed. Services, Calculations, Compiling, Version-Controls, etc. Memcache especially, being something that handles huge traffic is a highly distributed system, in itself, and its fun to learn new stuff related to it.

Coding: the coding style is very different from our day to day life coding, when working on big projects, design is extremely important, the code has to be secure, fast and scalable. Writing efficient code for such systems is not something that you learn conventionally. Facebook also has in-house tools for developments, even languages built upon one another. I use variants of c++, php, python, and some inhouse languages for coding.

Theoretical Stuff: It is great to actually use the concepts that you have learnt in practical world it is an experience to see the concepts fail because you forget to take care of certain factors (most likely scaling), then improve on them, and its amazing to see your ideas finally work and go live on such a huge scale. These are platforms where you really realize how awesome your courses were, and feel like digging up books again.

Hardware, Operating Systems, Replacement Policies, Automata, Intelligent Searches, Hashing Functions are all the things I am working closely with and Its simply superb to see all of this come to life.

It can be frustating and overwhelming at times though :P mainly because theories do not always correspond to practical situations because there are many factors at play. In all, its a great learning experience.

Random: Working for tech giants has many perks. PlayStations, Infinite Free food, couches, flexible working hours, game arena, TT/pool/foosball are some of the very few.

The best part is that you might sometimes get chance to work with/discuss strategies/have lunch with people whose books you read. Who are considered gods of their field, in some cases even the makers of that technology.

[Anil Shanbhag]

While we are on code complexity : have a look at  http://en.wikipedia.org/wiki/Cyclomatic_complexity

@Saif : You can share yours too :) 

--

Anil Shanbhag

[Saif Hasan]

Hi, I am sharing my previous internship experience.

Yahoo! Labs: 

In my second summer vacation I did internship at Yahoo! Labs Bangalore. Previously I didn't had any clue about difference between Labs and Development Intern. At Labs people just do interesting research and it is full of PHDs on the other hand Development team is full of people who do implementation for real life products.

My work at Yahoo was on Sentiment Analysis of Webpages, classifying webpage into positive and negative based on its content. This was really challenging problem. I had no previous background about machine learning concepts, but it was really interesting to work on it and I learned lot of things.

Furthermore implementing the code for testing huge amount of data in Hadoop was fascinating. TBs of data gets processed in couple of minutes. As a sophomore I witnessed tons of magic of Computer Science.

Bing, Microsoft:

In my third summer vacation I did internship at Microsoft Hyderabad in Bing team. My work was to develop a debugging tool for Bing search engine when something goes wrong.

Search engine is a very big stack which is very hard for one to understand completely. During my internship I came across almost every aspect and part of a real time search engine which was very interesting. Commercial production of code/software is very different from the ones we do in our course assignments.  Work was very exciting and fun . . (p.s. I was interested in search engines well before and I got to intern with a real one) . . . . 

Mentor System (for those who haven't been to any internship): It is not difficult to work with big companies and worry about whether you will be able to contribute well or not. Internee is provided with one mentor who makes internee familiar with concepts and development environment and help throughout the project.

You received this message because you are subscribed to a topic in the Google Groups "Web and Coding Club IIT Bombay" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wncc_iitb/0Z36JKY9lnY/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to wncc_iitb+...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Saif Hasan | Senior Undergraduate | Computer Science & Engineering | IIT Bombay

[Sanket Totala]

Hi,

I am doing my intern at Microsoft Hyderabad with Bing (it is a search engine like Google, for those who don't know :P)

My team is Global Relevance (we call it gr express). It deals with the creation of data (streams) used for ranking of a page.

It is after 6 hours of lecture by my mentor (about how search works) that I understood nothing except that millions of computations take place in milliseconds for a single query...

I am very lucky to have an awesome mentor, who keeps throwing his vast knowledge on me. My manager has said to me "Working less and learning more must be your motto". So, I am "working less and playing more" :P

Coming to my work, it is mainly to develop tools that are helpful in debugging if something goes wrong (for eg, the stream size reduces drastically, etc..)

I need to work with scope scripts (as explained by Ashwin previously) to gather data in required format, and integrate it with the present tools/create new ones (in c#). Finally, for displaying, SQL-BingDAT jobs (another internal tool) is used along with HTML. It has been an amazing experience till now. The only bad part is that we need to work on Windows and use its "tatti" command prompt.

Microsoft has an amazing way of getting work done (for interns): We need to fill up a commitment form (which describes all the work we are going to complete in the internship period). If we perform satisfactorily, we will be considered for PPO. So, all of us don't just chill off..

Last year, I interned with Amazon, Bangalore with the Affiliates team (to be specific, Fraud Detection).

The concept is that:

1. I can display add of Amazon on my website (called the Affiliate Program).

2. If someone visits my site, clicks on the add and but something from Amazon, I will get commission.

3. This is free money. People use unethical ways to earn money in this way. I have built such pages. An exapmle is: http://www.cse.iitb.ac.in/~sankettotala/cookie_stuff/

My work was to detect this "Fraud".

Though I did not complete my work, I was given a PPI (ya PPI, not PPO)

Ask the placement cell about PPI.

It was fun working with Amazon, but Microsoft is definitely better..

Hope you guys are having lots of fun as well

Thanks and Regards,
Sanket Totala
4rd Year Undergraduate
Computer Science Department
IIT Bombay

[Dilawar Singh]

I did my internship (and later job for some time) as 'some company' which developed firmware for some movie-camera making corporation. My task was to implement Kalman filtering to stabilize images in Video: so far they were using simple low-pass filter (moving average) for it. The data was coming from motion sensors (gyrometer and accelerometer) attached to the camera body. Hardware was a DINI board (multiple FPGA board) and we worked closely with HCL team. DE0-nano board has an accelerometer in it. You can play with it, though you have to write your algo in some HDL. We were using c++ on Multi-OS which was essentially a linux-core.

As usually only 10% people work in any organization, rest are there to do their time. Development work contains a core which is very simple and fundamental and lot of bull-shits/UI/APIs around it (they pay for doing it). For instance, core of spice is nothing but a sparse AX=B solver. Feel lucky if you get to work on this core, luckier if you get to improve it.

--

Dilawar

EE, IITB

[harshit mittal]

Elaborate the last couple of lines, please...

--

Cheers!

Harshit