Informative post

[Harshad Deshmukh]

Hi,

Came across this. Hope it proves useful. 

http://royal.pingdom.com/2011/03/08/the-big-data-cook-book/

~ Harshad

[Vineet Pandey]

Informative.

Thanks for sharing.

vineet

[Vineet Pandey]

Not the best time to say this, everyone would be busy with endsem madness, but do you think its time you guys put up a webpage and a blog with small updates, CSA site can be used. could be done in hols.

Vineet

[Aastha Mehta]

Another informative post: http://www.unhosted.org/

--
Aastha Mehta
Intern, NetApp, Bangalore
4th year undergraduate, BITS Pilani
E-mail: aast...@gmail.com

[Pranav Agarwal]

@Vineet: Me and Ambarisha are making an ASIG Document which will cover whatever we have discussed this semester. I guess, we will be done with it by Saturday. Also, we can upload it on CSA site as well for others to download.

Pranav

[Vineet Pandey]

That's great. A small blogpost would help along with it since document requires greater attention span. 

vineet

[Pranav Agarwal]

Sorry but this may be a lame question.. but what does this mean ?

"In standard websites, user data is stored in the same server as the application.On unhosted websites, data is taken away from app servers and lives on the data node you login with." - taken from this

Pranav

[Pranav Agarwal]

Hey all, 

I was finally able to draft a small document on Fast File System. Other topics such as Log Structured File System and BFS will be documented later. In the mean time, if you like this document, feel free to spread it around. Also, if there are certain things which are wrongly written, do let me know. In addition, i have also attached the papers. I had highlighted a few things when i was reading it. Hope it helps.

Pranav

[Aastha Mehta]

Hi Pranav,

I have a few questions to ask, regarding things that I did not understand from your document:

1. Observations on page 3 - what are the file systems you are talking about. Are 4096 and 1024 the fragment sizes which have been used in some typical filesystems?
2. On page 4 - What do you mean by file being fragmented or file containing fragmented blocks? How is it different from the fragmentation of the physical blocks on disk?
3. Could you throw some light on how cylinder groups are made? I mean, are there fixed no. of cylinders/tracks allocated in each group or are they different? Is a group just another term like tracks, sectors, logical blocks, etc.? What is the typical size of each cylinder group? I wish to understand this, so that I can understand why a global policy forces a long seek on unavailability of sufficient blocks within it's current cylinder group.
4. The file layout policies that you discussed on page 5 - are they generic policies implemented in other non-FFS filesystems?
5. What are the problems in using the policies with larger files that you have mentioned on the same page? I vaguely understand the solution that you gave for the problem, but could not get what problem was it trying to address.

A really nice piece of work! Thanks for putting it up on the group.

Regards,

Aastha.

[Pranav Agarwal]

Hey,

Good to see, someone read it. Sorry, i didn't write everything in proper sentences in the doc as i thought points would have been sufficient. 

Back to Problems:

1. 4096/1024 is a file system with 4096 byte blocks and 1024 byte sized fragments. basically, one block can have 4 fragments. So the observation was that although the wasted space is the same in both a 4094/1024 file system and a simple 1024 block sized file system but the bandwidth in the former is higher compared to the latter.

2. The basic idea is that when someone writes a file, a few blocks (say) are allocated. Now that you want to append something to the file, the expression "File Contains no fragmented Blocks" means that whatever blocks were assigned to the file, if it is full or not. Fragmentation is always at the block level. Fragmentation of files, i suppose, means nothing.

3. AS far as i had read, Cylinder Groups are static in number and they are defined by the file system during its installation. Yes, there are fixed number of cylinder groups in a file system and fixed number of cylinders in each cylinder groups.You can check your system config using the linux cmds.

4. These layout policies seems very manaegable and so i think that such a scheme should be used in one way or the other on any other file systems but i am not very sure of this. I'll search about these and let you know.

5. "The problem with allocating all the data blocks in the same cylinder group is that large  files will quickly use up available space in the cylinder group, forcing a spill over to other areas. Further, using all the space in a cylinder group causes future allocations for any file in the cylinder group to also spill to other areas. Ideally none of the cylinder groups should ever become completely full. The heuristic solution chosen is to redirect block allocation to a different cylinder group when a file exceeds 48 kilobytes, and at every megabyte thereafter."

This is the statement taken directly from paper.

Pranav

[Aastha Mehta]

Ah, now this clears up those questions. Pranav, could you modify the documentation to include these answers as well?

Thanks,

Aastha.

[Pranav Agarwal]

Changes made :-)

Pranav

[Vineet Pandey]

Interesting: http://arduino.cc/en/

The cheapest model costs 1360 in India (check their page for India sellers). 

vineet

[Ramakrishna Reddy Kunreddy]

thanks for posting .. yes we have few arduino boards in LAB !!

--
Ramakrishna Reddy K, 

M.E. Computer Science,

Android Mobile Computing Research Group Lead,

SDETU,BITS-Pilani,Rajasthan.

[Rishabh Mehrotra]

We bought Arduino last year for the Omniscient Bot project. That Arduino board cost us around 1300 and the Bluetooth around 800. In the market survey we did during APOGEE the Bluetooth-Arduino was nothing less than 4k.

Different versions of the boards are available in the market. Even the Acyut guys sell their Gorduino board around the same cost.

--
Rishabh Mehrotra
BITS Pilani.

[Pranav Agarwal]

I don't know a lot about Arduino boards but i guess i found one of the applications.

http://www.howtogeek.com/news/wi-fi-speakers-stream-music-anywhere-in-your-house/6233/

Pranav