Image recognition system in hadoop

[aSh wIn]

Hello all,

My name is ashwin. now im currently doing my research on Performance analysis of biometric data processing by distributed framework(cloud)... .its like image recognition system using hadoop and map reduce...i need to store biometric data in hadoop and write my java program on top of it to do the image recognition system. I want to know how this can be done sir...is it possible to use HIPI to do this?.. Im really appreciate your help guys.. Please reply me ASAP because my project completion time is just around the corner... Thank you guys...

[Pete Warden]

Hi Ashwin,

                I've successfully used HIPI to image recognition across hundreds of millions of images, so it's definitely possible. What problems are you hitting when you're working with the framework?

Pete

On Fri, Aug 1, 2014 at 10:55 AM, aSh wIn <cvaa...@gmail.com> wrote:

Hello all,

My name is ashwin. now im currently doing my research on Performance analysis of biometric data processing by distributed framework(cloud)... .its like image recognition system using hadoop and map reduce...i need to store biometric data in hadoop and write my java program on top of it to do the image recognition system. I want to know how this can be done sir...is it possible to use HIPI to do this?.. Im really appreciate your help guys.. Please reply me ASAP because my project completion time is just around the corner... Thank you guys...

--

Check out Jetpac City Guides iPhone app - Just launched!

CTO Jetpac

Follow me on twitter @petewarden

[aSh wIn]

Hi pete,

Thank you for the reply. I havent try HIPI yet. Im just wanted to know whether is it possible to do image recognition with HIPI... My main idea is, normally we do image recognition system using matlab or opencv but that done in local machine..but now i want to the same thing with hadoop distributed environment. Finally,  i have to analysis the performance (efficiency,speed) of recognition system in both local machine and hadoop environment.. This is my main objective. Do you have any idea on how to perform this? Thank you sir. 

--

Best regards

Somatheran Balakrishnan

Somatheran Balakrishnan @ Ashwin

Student@Multimedia University

Tel: 014-9312474 | E-mail: cvaa...@gmail.com / @yahoo.com

  

[Pete Warden]

Ashwin, that sounds a lot like a homework problem! We can definitely help you out with concrete issues you hit while trying to code up an application, but there isn't much I can offer unless you can give a clearer statement of what you need.

[aSh wIn]

Thank you Pete.. Actually my main objective is to develop image recognition system in hadoop distributed environment and analyze it's performance and compete it with local system. I want to know how this can be done? That's why I need your help Pete. Thank you. :)

[Aswin Suresh]

Hi Ashwin and Peter, this is Aswin Suresh. I am doing a project on remote satellite image analysis using mapreduce and HIPI. Though I have the code for my logic, I dont know how to implement it in HIPI and MapReduce. Like, I need help in syntax part, how to use mapreduce and how to connect all this to my logic. Could anyone of you please help me on this regard, as I am left with very less time for the completion of the project. Thank you.

[Aswin Suresh]

Hi Ashwin and Peter, this is Aswin Suresh. I am doing a project on remote satellite image analysis using mapreduce and HIPI. Though I have the code for my logic, I dont know how to implement it in HIPI and MapReduce. Like, I need help in syntax part, how to use mapreduce and how to connect all this to my logic. Could anyone of you please help me on this regard, as I am left with very less time for the completion of the project. Thank you.

On Friday, August 1, 2014 at 11:25:49 PM UTC+5:30, aSh wIn wrote:

[Chris Sweeney]

Hi Aswin,

This email group should be used for questions related specifically to HIPI. You are asking about basic MapReduce and Hadoop syntax, and I would suggest you try some Apache tutorials for Hadoop or email one of the relevant Apache email lists.

Chris

[Lingbo Wang]

Hi Ashwin,

We are using HIPI now to do a project, and we face the same problem. We want to match one images with many images to see which one matches the best. We now have the match algorithm and we have HIPI, but we don't know how to implement the map function, especially how to pass two images to the map function. Can you help us?

Best,

Lingbo

[Dhanashree Jadkar]

I have error when I m using covar tool of HIPI

[root@quickstart tools]# ./covar.sh hdfs://quickstart.cloudera:8020/user/cloudera/sampleimages.hib hdfs://quickstart.cloudera:8020/user/cloudera

Running compute mean.

17/03/22 22:38:22 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032

17/03/22 22:38:22 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.

17/03/22 22:38:22 INFO input.FileInputFormat: Total input paths to process : 1

17/03/22 22:38:22 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/root/.staging/job_1490160386159_0006

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1

at java.util.ArrayList.elementData(ArrayList.java:400)

at java.util.ArrayList.get(ArrayList.java:413)

at org.hipi.imagebundle.mapreduce.HibInputFormat.computeSplits(HibInputFormat.java:98)

at org.hipi.imagebundle.mapreduce.HibInputFormat.getSplits(HibInputFormat.java:177)

at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305)

at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322)

at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)

at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)

at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325)

at org.hipi.tools.covar.ComputeMean.run(ComputeMean.java:44)

at org.hipi.tools.covar.Covariance.run(Covariance.java:116)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

at org.hipi.tools.covar.Covariance.main(Covariance.java:135)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)